Effective animal advocacy bottlenecks surveys

By Jamie_Harris @ 2021-01-13T13:38 (+31)

This is a linkpost to https://www.animaladvocacycareers.org/post/effective-animal-advocacy-bottlenecks-surveys

Abstract

A better understanding of the problems that limit the efficiency and total effects of the animal advocacy movement is useful for advocates, movement-builders, and individuals making decisions about careers that help animals. We therefore ran a survey of leadership and hiring managers at effective animal advocacy nonprofits and a survey of researchers, grant-makers, plus others working on “meta” and movement-building services for the movement. Lack of funding was identified as a key bottleneck in both surveys, with lack of (qualified and capable) applicants for paid roles not far behind. Other results suggest that a high threshold is required for individuals to help animals more by choosing careers focused on donating money to nonprofits rather than donating time and labour. Organisations face the most substantial hiring difficulties for roles focusing on leadership or senior management, fundraising or development, and government, policy, lobbying, or legal tasks. There are few differences by role type in terms of retaining staff, though fundraising and development stands out as slightly more difficult than average. Headhunting and training for current staff of animal advocacy organisations were commonly suggested as possible solutions to talent bottlenecks. Some differences emerge when comparing responses from organisations based in the global South to those based in the global North, organisations operating primarily in Asia to those operating primarily elsewhere, or with large numbers of employees to those with small numbers of employees.

Introduction

Shortly after founding AAC, we conducted a brief survey about talent bottlenecks. That survey substantially informed the work that we carried out in 2020, so we were keen to improve the question wording, seek input from a wider variety of stakeholders, and update the results.

We therefore ran two surveys with partly overlapping questions:

• A survey of leadership and hiring managers at effective animal advocacy nonprofits (hereafter the “direct work” survey).
• A survey of researchers, grant-makers, plus others working on “meta” and movement-building services for animal advocacy (hereafter the “meta” survey).

The questions wordings were optimised for usefulness in AAC’s own decision-making.[1] However, we hope that the results can also be useful to:

• Others thinking about providing various meta and movement-building services for the animal advocacy and effective altruism communities.
• Participating organisations, who may better understand which difficulties are common and which unusual, so that they can better coordinate with other organisations and understand where it would be helpful to seek or offer advice.
• Individuals making decisions about careers that help animals, who may gain a better understanding of various factors that would affect their own career strategy, such as the impact potential of different career paths and where their comparative advantage lies.

Methodology

For the direct work survey, we contacted organisations that met the following inclusion criteria:

• Currently or formerly recommended (“standout” or “top”) by Animal Charity Evaluators at the time that we sent out the survey (16 contacted),
• Not recommended by ACE but granted $50,000 or more by the “Farm Animal Welfare” focus area of Open Philanthropy at the time that we sent out the survey (35 contacted),
• Neither of the above two criteria but very high alignment with the ideals of effective animal advocacy, i.e. the overlap of the effective altruism and animal advocacy movements (5 contacted).
   

We also applied the following exclusion criteria:

• If their work primarily focused on “meta” services to other nonprofits,[2]
• If effective animal advocacy was a low focus of the organisation,[3]

For the direct work survey, we contacted 56 organisations and received 39 responses from 39 different organisations (70% response rate).[4] We asked for one representative response from each surveyed organisation, noting that this “should be filled out by someone in a top-level leadership role (e.g. CEO, ED, COO, VP) or a hiring manager/human resources staff member.” All except three respondents identified themselves as being in a “leadership (e.g. CEO, ED, COO, VP)” role.[5] You can see the questions used in this survey here.

We conducted subgroup analyses for nonprofits with each of the following focuses:

• Corporate/producer welfare campaigns (n = 21),
• Political campaigns or engagement (n = 15),
• Individual diet change (n = 12),
• Institutional/corporate animal product alternatives outreach (n = 11),
• Groups that focused on animal product alternatives capacity-building (n = 3),[6]
• Other actions (n = 22).[7]

Most groups were included in more than one category. A group was only included if it seemed to spend a substantial amount of resources on this type of action.[8] This involved some subjective judgement calls; certain organisations did not clearly report their spending by intervention type.

We conducted several other subgroup analyses:

• Organisations based in the global South (n = 6) compared to those based in the global North (n = 33).[9]
• Organisations that work primarily in Asia (n = 9) compared to those that work primarily elsewhere (n = 30).
• Organisations that mostly target one country (n = 17) compared to those that work more internationally (n = 22).
• Organisations ever recommended (“standout” or “top”) by Animal Charity Evaluators (n = 14) at the time that we received the responses[10] compared to those not yet recommended (n = 25).
• Small organisations that told us that they had 10 or fewer “full-time equivalent, paid staff”[11] (n = 17) compared to medium-sized organisations that told us they had 11 to 50 (n = 14) and large organisations that told us they had 51 or more (n = 8).

Given the small sample sizes involved and very large number of comparisons being made, we decided not to conduct formal significance tests to check whether the differences between subgroups were statistically significant. The results of the various subgroup analyses can be seen on this spreadsheet. That spreadsheet also includes a column where we treat nonresponses to a question as the lowest possible score (0, 1, or no selection) rather than just excluding non-responses from the analysis.

The main results reported below are either the mean score from respondents (for quantitative questions) or the percentage of respondents that selected a particular option. We were concerned that treating all responses to the survey equally (taking the mean of responses without any adjustment) might present an unrealistic account of the movement’s needs and bottlenecks. To address this concern, we also tried several weighting systems for quantitative questions:

• “Weighting 1” accounted for the size of the organisations.[12]
• “Weighting 2” and “weighting 3” accounted for both size and ACE recommended status; weighting 2 placed twice as much weight on ACE Top Charities as charities that have never been recommended by ACE, whereas weighting 3 placed 10 times as much weight on them.[13]

These weighting systems made surprisingly little difference to most questions.[14] Hence, we decided to only include these weighting systems on the subgroup analyses spreadsheet, rather than including them in the main results tables below, as we had initially intended.

For the meta survey, we used less formal inclusion or exclusion criteria, contacting organisations working on research, grant-making, and other movement-building services whose input we expected to be useful. We did not suggest that we were hoping for the response to be representative of their organisation, though in most cases we still only asked for one response from each organisation, to reduce the time spent filling out the survey. We contacted 21 organisations and received 16 responses from 13 different organisations (62% response rate). You can see the questions used in this survey here. Given the small sample size, we did not conduct any subgroup analyses or apply any weightings.

Since a key goal of the surveys was to see how respondents’ answers differed from our expectations, we did not include our own responses in either survey; instead we jotted down predictions and rough notes and compared the results to those. We did not formally pre-register our analysis plans, but did informally plan most of our analysis in advance.[16] We decided not to provide the full dataset, in order to protect the anonymity of participants, though we are open to requests to conduct additional analyses.

Results and discussion

In the writeup below, we include qualitative comments about differences between subgroups or weighting systems that we thought were noteworthy, though you may prefer to ignore these and look at the results of the subgroup analyses for yourself. As well as a list of general limitations at the end, we note limitations that apply to particular questions as we go along. A single asterisk (*) signifies that this was a concern we had written down before seeing the results. Two asterisks (**) signify limitations that we had not necessarily thought of but that were highlighted by respondents to the questions themselves.[17]

The importance of different bottlenecks

We asked direct work respondents the following question: “To what extent does each item below limit your organisation’s efficiency or impact?” We asked meta respondents the same question but replaced “your organisation’s” with “the EAA community’s,” which was defined as “the group of people and organisations who are focused on maximising their positive impact for animals.”[18] We offered them the following options: “1 Not at all”, “2”, “3 Somewhat”, “4”, “5 Very much”.

Here are the average scores:

Differences by subgroups and weightings:

• Under the weighting systems, “Difficulties with internal culture” and “Difficulties coordinating internally” became more important as limitations, whereas “Unfavourable attitudes among targets of your advocacy, e.g. companies, government, consumers” became less important.
• The average rating of items was higher for organisations working primarily in Asia (3.0) than elsewhere (2.5). The largest increases were for “Lack of (qualified and capable) activists and volunteers,” “Lack of (qualified and capable) applicants for paid roles,” and “Lack of public awareness of key issues,” which were rated as the most substantial limitations on organisations’ efficiency or impact (4.2, 3.9, and 3.8, respectively, compared to 3.7 for “Lack of funding”).
• Among organisations focusing on corporate/producer welfare campaigns, funding was slightly less important as a limitation and was narrowly overtaken by “Lack of (qualified and capable) applicants for paid roles,” “Lack of (qualified and capable) activists and volunteers,” “Lack of public awareness of key issues, e.g. knowledge about factory farming,” and “Unfavourable attitudes among targets of your advocacy, e.g. companies, government, consumers).”[20]
• The average score for bottlenecks was higher for organisations focusing on “Individual diet change” (2.9) than for the whole sample (2.6). The greatest increases were for “Lack of funding”[21] and “Lack of public awareness of key issues, e.g. knowledge about factory farming.”
• Among organisations focusing on institutional/corporate animal product alternatives outreach, “Unfavourable attitudes among targets of your advocacy” were rated as less important limitations.
• Surprisingly, the ranking of items from most to least limiting was similar for both the global North and the global South.[22]
• There were a number of differences between organisations working mostly in one country and organisations working internationally. For example, organisations working in one country reported being more limited by funding, lack of (qualified and capable) applicants, and “Unfavourable attitudes among targets of your advocacy,” but less limited by “Difficulties coordinating internally.”

The following limitations should be borne in mind:

• **Several respondents suggested additional factors that we did not ask about.[23]
• *Some of the options are especially ambiguous or broad. For example, several respondents noted that they were unsure how to interpret “difficulties with internal culture.”
• *Logically, “Unfavourable attitudes among targets of advocacy, e.g. companies, government, consumers)” should presumably be rated highly by everyone; if the targets of advocacy have favourable attitudes, then in most cases, no advocacy would actually be needed.[24] Although this was one of the highest-rated factors, the fact that it was not rated as much higher than other factors should encourage some scepticism of the results.
• **Several meta respondents noted that their answers would vary for different regions of the world.
• Some responses could reflect temporary conditions rather than medium-term problems, such as coordination or decision-making difficulties arising from COVID-19.

The value of donations relative to high-quality direct work

As the above results show, there are clearly many different types of bottlenecks that can affect the animal advocacy movement. However, one particularly salient tradeoff in many decisions — both for organisations’ leaders and for individuals seeking to maximise their positive impact for animals over the course of their careers — is the value of donations relative to high-quality direct work.

We asked direct work respondents the following question: “Imagine that someone has been working for 10 years building up experience and expertise that would make them an excellent candidate for one of the roles that is *hardest to hire for* in your organisation. Would you be more excited about that person applying for one of those roles at your organisation, or donating money to your organisation that was the equivalent of 50% of the salary of that role?” We then asked them the same question again but replaced “one of the roles that is *hardest to hire for* in your organisation” with “*a campaigns, corporate engagement, or volunteer management* role in your organisation.”[25]

For both questions, we offered them the following options: “Much more excited about them applying” (coded as 1), “Somewhat more excited about them applying” (2), “Roughly similarly excited either way” (3), “Somewhat more excited about them donating” (4), and “Much more excited about them donating” (5).

Here are the average scores:

These results seem like a vote in favour of careers in direct work in general, even for role types that are not necessarily “hardest to hire for.” We note below that the average salary seems to be around $50,000 or so in animal advocacy nonprofits in the global North, though the average salary might be higher for roles that are “hardest to hire for,” such as leadership and senior management roles. So you could interpret these results as suggesting that organisations would rather receive one additional very-high quality applicant for the roles that are hardest to hire for than receive $25,000 or more (each year for the length of time that the applicant might otherwise have been employed for).

The following limitations should be borne in mind:

• The needs of the community are likely to change substantially in 10 years, so if someone were to pick an option based partly on these results, they might find that, with hindsight, this was the wrong decision.
• *A lot of people won’t have good personal fit with earning to give or with the various roles in nonprofits that are available to them, or they may have already developed expertise relevant to one of these paths. In no case should a career decision depend solely on the relative need for funding or high-quality candidates of a particular role type.
• *Someone with 10 years’ worth of relevant experience to a role type that is “hardest to hire for” is probably pretty close to the ideal candidate for a hire.
• Donating 50% of a salary is not the peak of achievement that we might reasonably expect from a relatively competent and privileged individual whose main career goal is to earn to donate as much money as possible. The US Bureau of Labor Statistics lists many occupations with an “annual mean wage” well over $100,000 per year,[27] which compares to average salaries at animal advocacy nonprofits in the global North being around 50% of that (see below).
• *Given that the respondents have all themselves chosen to focus on direct work rather than on earning to give, we might expect that they would be more likely to overvalue direct work.
• *Respondents might not think that much about replaceability, so they might see a good candidate as contributing 100% of what a good candidate does, which would be misleading.
• *Respondents might not have thought about the many ways in which money could be transferred into higher talent (e.g. training, different kinds of recruiting, higher salaries).
• *Helping animals directly seems likely to be associated with kindness, altruism, and other positive characteristics, whereas earning lots of money doesn’t, even if it actually does a lot to help animals.
• **If individual donations are not an important stream of income for an organisation (e.g. because they rely on grants), then those organisations may have given a lower rating than they otherwise would.
• **Some respondents may have interpreted the question as referring to a one-time donation; our intention was for the question to refer to ongoing donations (equivalent to the length of time that they might otherwise have been employed for, for the sake of comparability), but this was not specified, for the sake of brevity.

As another way of understanding this trade-off, we asked meta respondents the following question: “Imagine an individual who is skilled and motivated enough to be a good (but not outstanding) candidate for roles in effective animal advocacy nonprofits. I.e., after a few applications, they are likely to secure a role, but they are not likely to be substantially better than the next best candidate, at least in their first paid role. How much money would you estimate that that person would have to be able to donate per year, on average, to effective animal advocacy nonprofits, to be indifferent (from an impact perspective) between focusing on a career “earning to give” vs. a career in animal advocacy nonprofits?”[28] The average answer given was $28,200, with a range from $100 to $90,000.[29]

The following limitations should be borne in mind:

• *This question seems especially vulnerable to minor changes in wording. For example, removing “but not outstanding” might have changed the answers quite a lot.
• *Two respondents explicitly noted that the question was difficult to understand. Other respondents probably also struggled or misunderstood our intentions but just didn’t tell us (or realise).
• *This is a very complex tradeoff with many relevant factors involved. Survey respondents may answer survey questions quickly and substitute complex questions for simpler questions that they can answer more easily.
• *Given the complexity of this question and the other questions that we wanted to ask the direct work respondents, we only asked this question to the meta respondents. These answers therefore reflect a narrow group of stakeholders with relevant perspectives on this question and come from a small sample size.
• Given the large range in the responses (three orders of magnitude), the answer to this question seems especially uncertain.

The importance of different talent bottlenecks

We asked direct work respondents: “How difficult is it to *hire* high-quality candidates for the following categories of roles or types of expertise?” We then asked them the same question again, where “*hire*” was replaced with “*retain*”. We offered them the following options: “1 Not at all”, “2”, “3 Somewhat”, “4”, “5 Very much”.

Here are the average scores:

Differences by subgroups and weightings:

• Under the weighting systems, “Fundraising or development” and “Government, policy, lobbying, or legal” became more important as hiring difficulties, with similar scores to “Leadership or senior managers.” Meanwhile, “Natural sciences,” “Research,” and “Middle or junior managers” became less important.
• Under the weighting systems that reflect ACE recommendations, “Fundraising or development” stands out more from the other categories as being notably more difficult to retain staff in.
• Counting non-responses as 1s rather than excluding them reduces the apparent importance of difficulties hiring and retaining “Natural sciences” roles.[30]
• Among organisations focusing on “Corporate/producer welfare campaigns” and “Political campaigns or engagement,” roles in “Campaigns, corporate engagement, or volunteer management” and “Government, policy, lobbying, or legal” became more important as hiring difficulties.[31]
• Organisations focusing on “Individual diet change” reported higher difficulties in both hiring and retention with “Leadership or senior managers,” “Marketing or communications,” and “Government, policy, lobbying, or legal” roles.
• Animal product alternatives capacity-building nonprofits gave especially low average scores for retention difficulties (average 1.2 across all categories). “Fundraising or development” stood out as causing the largest hiring difficulties.
• Compared to the respondents in the global North, those from the global South reported greater difficulties in both hiring and retaining staff in “Leadership or senior managers,” “Marketing or communications,” and “Fundraising or development” roles.[32]
• The average rating of difficulties in both hiring and retention was substantially higher for organisations working primarily in Asia (3.6 and 3.2) than elsewhere (2.6 and 2.0).[33] The only categories that did not increase substantially were “Operations, administration, or HR” and “Other technical skills.”
• Similarly, organisations working in one country reported larger hiring difficulties than organisations working internationally across many role types. This partly reflects the substantial overlap between the category for organisations working primarily in one country and the category for those working primarily in Asia. Again, operations and “other technical” roles did not have much higher scores; this time, research and “Fundraising and development” were also exceptions.
• Across the three size categories, there were few consistent trends in terms of particular categories of role becoming more or less difficult to hire and retain. Middle or junior managers seemed to become less difficult to hire and slightly less difficult to retain as an organisation’s size increases.

The following limitations should be borne in mind:

• **We provided the guidance “Feel free to skip any items that do not apply to your organisation,” but respondents still seem to have taken very different approaches when they had (so far) had no experience with hiring or retaining someone in that type of expertise.[34] This will introduce additional random noise to the data that makes the responses more difficult to interpret.
• **Three respondents offered comments that suggested that particular sub-categories of roles were especially difficult to hire for or retain.[35]
• **Two respondents offered comments noting that the difficulties vary by region because campaign tactics are less socially accepted in some countries than others.[36]

We didn’t explicitly ask respondents whether they saw hiring or retention as more of an issue, but the survey provides some evidence that respondents see hiring as more of an issue.[37]

Allocation of talent by sector

The above results focus on effective animal advocacy nonprofits. However, working in these nonprofits is certainly not the only way that an individual can help animals over the course of their career. Hence, we also asked meta respondents how they thought that “the next 100 highly motivated, highly competent individuals looking for careers where they can have a positive impact for animals” should divide their efforts between different broad career paths.[38]

Here are the average percentages given for each sector:

The following limitations should be borne in mind:

• **Four respondents noted that there were instances of overlap between the sectors.[39]
• **The optimal distribution might vary substantially by region.[40]
• **The optimal distribution might vary by whether the individuals making career decisions are in the early or late stages of their career.[41]
• The optimal distribution might depend substantially on the extent to which the individuals making career decisions are “highly motivated” and “highly competent.”[42]
• *Since the meta respondents were all themselves working in nonprofits, we might expect that they would have an inflated sense of the importance of “working at existing nonprofits” and “setting up nonprofits.” Similarly, since many respondents were researchers, we might expect them to have an inflated sense of the importance of “Academia.”
• *The question doesn’t account for differences in personal fit or other career strategy considerations, so higher percentages do not necessarily indicate the best option for any particular individual.
• *Some roles might be important but only needed in small numbers, so higher percentages do not necessarily indicate higher impact potential.
• *There is no clear source on the actual plans and decisions of “highly motivated, highly competent individuals looking for careers where they can have a positive impact for animals” which we can compare to the answers to this question.
• The EA Survey provides data on the “broad career path (s)” that individuals identifying with the effective altruism community “are planning to follow.” The effective altruism and animal advocacy communities are at least partly separate, and the methodology on that survey is different to methodology here; for example, respondents could select more than one career path. The most obvious trends are that respondents to the EA survey seemed to be more favorable towards earning to give and academia than the respondents to our survey.
• We have asked applicants to our one-to-one careers advice calls service about their plans, using similar categories as here. At the time of checking, with approximately one week left before the final deadline, the following percentages of respondents selected “High interest/priority” for each of the following: “Working directly for a nonprofit,” 76%; “Policy and politics,” 43%; “Work for a for-profit animal-free food company,” 42%; “Earning to give (maximising income in the for profit sector in order to donate to effective animal advocacy organisations),” 22%; “Setting up a new charity,” 21%; “Relevant research in academia,” 36%; “Related legal work, e.g. animal welfare law,” 28%; “Setting up new companies producing or selling alternative foods to animal products,” 18%. However, our advising service is unlikely to be very representative of “highly motivated, highly competent individuals looking for careers where they can have a positive impact for animals” and, again, participants were able to select multiple options. Earning to give does not seem to be overrepresented among our applicants; if anything, it seems to have a smaller markup relative to the average preferred percentages by the meta survey respondents than some of the other sectors. “Working at existing nonprofits” / “Working directly for a nonprofit” has the smallest markup, but still the greatest interest, by quite a large margin.

Organisational questions

We asked respondents to the direct work survey several organisational questions. Here are the average answers:

There were substantial differences between organisations based in the global North and the global South. Average salaries were $52,039 in the former[47] and $10,122 in the latter. Estimated hiring costs were $2,548 in the former and $377 in the latter. The average spend on staff development and training was apparently higher in the global South ($2,133.3) than the global North ($1,996), though with one outlier excluded (which we expect is a mistake[48]), the average for the global South is only $560.

Though the samples and methodologies do not match up exactly, we can compare these figures for respondents from the global North to figures from research into organisations in other contexts. In each case, the numbers seem surprisingly close:

• We can compare the average estimated salaries for respondents ($51,668) to the average salary in the US ($53,490).
• We can compare the average estimated hiring costs ($2,548) to estimates from academic studies in other contexts, which range from thousands to hundreds of thousands of US Dollars, though these include various costs not included by our survey question; the most relevant comparisons also seem to be in the low thousands.[49]
• We can compare the average estimated spend on staff development and training ($1,996) to “U.S.-based corporations and educational institutions with 100 or more employees” surveyed by the Training magazine: average $1,286 in 2019, or $1,889 for nonprofits specifically, and lower averages in previous years.[50]

The following limitations should be borne in mind:

• *Only 17 organisations were categorised as mostly targeting one country. Some organisations explicitly noted to us that they were unsure whether to respond with averages across all countries that they work in or just the country where their headquarters is based. This might affect results substantially, especially for organisations that operate in both the global North and global South.
• *We didn’t ask respondents to look up organisational records and calculate accurate figures. Hence, respondents likely guessed the figures — the answers should be treated as ballpark figures not as accurate answers.
• **A few respondents were reluctant to estimate spending on staff development and training because their organisation did not separately budget for these activities.
• **Some of the respondents from smaller organisations noted that hiring and training costs were low at the moment because they had so far hired “skilled people with whom we have previously worked” or similar.
• *The guidance of the hiring costs question directed participants to only account for direct hiring costs. However, modelling by researchers suggests that adaptation costs account for a much larger proportion of hiring and turnover costs — two-thirds or more of the total costs, in the identified studies that measured such costs.
• Whether these results seem close to relevant comparison groups might depend substantially on the chosen comparison groups.
• *There could be substantial variation by role type, but we didn’t ask about this specifically.

Possible solutions

We asked respondents to both surveys: “Which of the following specific interventions would you be most optimistic about Animal Advocacy Careers offering? (Select up to 3)”

For AAC, these results updated us most substantially towards being more favourable towards headhunting and less favourable towards creating new skills profiles or improving our existing ones.

Differences by subgroups and weightings:

• Respondents from smaller organisations, organisations based in the global South, focusing primarily on Asia, mainly working in one country, or not yet ACE recommended selected various forms of training relatively more frequently than their comparison groups and selected headhunting and a job board relatively less frequently. There is substantial overlap between some of these groups.
• Small organisations paid similar salaries to large organisations (average $51,094 compared to $51,389) but spent less on training ($1,220 vs. $3,288) and on hiring new staff ($1,496 vs. $4,483).

The following limitations should be borne in mind:

• *Every intervention that would directly and immediately feed into organisations’ operations was selected by 24% or more of the direct work respondents to this question, whereas few interventions that had a more indirect path to impact (e.g. via supporting potential applicants rather than current employees) were selected by this many. This could plausibly be because the usefulness of direct interventions is more apparent to the respondents, rather than because such interventions would actually be more useful, all things considered.
• *The “meta” respondents may be inclined to rate interventions that 80,000 Hours offers currently (one-to-one career advising, a job board, online skill profiles or careers profiles, and headhunting) more highly, either because their expectations of AAC are generally informed by the precedent of 80,000 Hours, or because they are confident in 80,000 Hours’ impact evaluation and decision-making.[51]
• **Three respondents commented that their interest in these services would depend on whether they were sufficiently tailored to region-specific requirements (this may apply to the next question too).

We asked respondents to both surveys slight variations on the following question: “If AAC could provide all-expenses paid, high-quality training for staff members in effective animal advocacy nonprofits whose job descriptions refer primarily to tasks in one of the following categories, which type of training would you be most excited about?”[52]

There are some confusing results here regarding management and leadership. Direct work respondents selected training for leadership and senior managers most frequently but training middle or junior managers nearly least frequently. Meta respondents did almost the exact opposite.

Differences by subgroups and weightings:

• Looking only at those 14 direct work respondents who selected “A training course in relevant skills for effective animal advocacy organisations’ staff” as one of the interventions that they would be most optimistic about AAC offering, 6 (43%) selected “Leadership or senior managers” — no other option was selected by more than 2 of these respondents.
• There was more interest in training in “Government, policy, lobbying, or legal” in the global South than the global North.
• The selections of training in “Fundraising or development” came exclusively from organisations that have not yet been recommended by ACE.
• Larger organisations selected training for “Leadership or senior managers” more frequently and for “Fundraising or development” less frequently.

The following limitations should be borne in mind:

• The most popular responses of the meta respondents all correspond to issues that AAC has written about in our initial talent bottleneck survey in 2019 and in our skills profiles. We expect that the meta respondents are more likely to have read AAC’s previous content than the direct work respondents, so their answers seem more likely to have been influenced by our previous writings on this topic.
• Since the respondents to the direct work survey were mostly “Leadership or senior managers” themselves, they may have an inflated sense of the importance of this work.[53]

General limitations

• *There are lots of plausible variations in wordings that we could have offered for these questions and small changes in wordings may have made a large difference to the results.
• **These surveys (especially the direct work survey) won’t necessarily identify difficulties facing not-yet-existing organisations that could be highly impactful.
• There is substantial overlap between some of the subgroups analysed. For example, most of the included organisations that work primarily in Asia also work primarily in one country. Given the many different possible dependent variables, we decided not to conduct regression analyses that would account for these associations between the characteristics of responding organisations.
• *We chose organisations to invite based mostly on the evaluations of Animal Charity Evaluators and Open Philanthropy. There is substantial overlap in the views of these two organisations. If you disagree with those views, then you might think that the results of this survey do not accurately represent the most important bottlenecks of the animal advocacy community.
• If you think that some subgroups are especially important (e.g. organisations working primarily in Asia, ACE recommended charities, or organisations working on animal product alternatives capacity-building), then you might think that we made insufficient efforts to survey groups that fit into that subgroup, or that the responses from that subgroup are more important than the overall results.
• *We conducted a separate survey for meta organisations and excluded several other research, capacity-building, or meta organisations from our direct work survey due to a low focus on effective animal advocacy. These groups could have been included; this may have affected the results.
• We have used an informal methodology to analyse the results, rather than formal statistical comparisons (e.g. t-tests) between respondents or subgroups.
• *Given that the subgroup analyses were conducted manually using Google Sheets, there is some risk of human error.
• *Given that we do not provide the full dataset (in order to protect respondents’ anonymity) or detail the exact steps taken in our analysis, our analysis cannot be replicated or checked by independent researchers.
• We regret not pre-registering our analysis plans. You might think that our analysis is biased somehow because we did not do this.
• *Our direct work survey mostly received responses from leaders at organisations. If we had also sought responses from individuals with different roles and perspectives (e.g. HR staff and hiring managers, middle or junior managers, entry-level employees in other role types, jobseekers, or external observers) we might have received different answers. These answers might plausibly have been more accurate.
• *As is suggested by the responses to the question about allocation of talent by sector, nonprofits are not the only key stakeholders in the effective animal advocacy community. Surveys of other relevant stakeholders, such as animal product alternatives companies or academics doing adjacent work, would presumably have produced very different responses.
• It is possible that, by publicising our previous research findings, we have influenced the results of the questions that we asked in these surveys.
• The sample size is quite low, especially for the meta survey and for some subgroup analyses of the responses to the direct work survey.
• Given that our organisation is called Animal Advocacy Careers, social desirability bias and response bias may have led to artificially high ratings for the importance of talent bottlenecks relative to other types of bottleneck.

Many of the limitations listed here could be influenced by our decisions in designing and running the surveys. So feedback on these limitations and decisions is especially welcome in the sense that it might help us to improve surveys in future years.

Footnotes

[1] In order to decide where to focus its interventions, Animal Advocacy Careers (AAC) needs to understand what the largest bottlenecks are in the farmed animal movement, i.e. the problems that most substantially limit its efficiency and total effects. We are most interested in understanding how different sorts of talent bottlenecks compare, but information about the importance of talent bottlenecks relative to other issues sheds some light on the usefulness of AAC’s work in general, as well the sorts of advice that we should give to individual job seekers.

[2] In most cases where this was the only reason for exclusion, these organisations were invited to participate in the “meta” survey instead.

[3] For example, many universities were excluded, where it would have been difficult to get survey results that had much relevance to their work on farmed animals. Some groups were excluded if they seemed to be primarily animal agriculture industry bodies; we expected these organisations to have very different needs and difficulties, though there were many borderline cases.

[4] The direct work respondents that gave us permission to share their organisation’s name were: ACTAsia, The Albert Schweitzer Foundation, Anima International, Animal Equality, Animal Kingdom Foundation, Animal Nepal, Animal Rights Center Japan, The Aquatic Life Institute, Brighter Green, Compassion in World Farming USA, Dyrevernalliansen, Environment & Animal Society of Taiwan (EAST), Equalia, Essere Animali, Eurogroup for Animals, Fair-Fish International Association, Fish Welfare Initiative, The Good Food Institute, The Greenfield Project, The Humane League, The Humane Society of the United States Farm Animal Protection, Mercy For Animals, New Harvest, People for Animals Uttarakhand, ProVeg International, Sentience Politics, Sentient Media, Sinergia Animal, Sociedade Vegetariana Brasileira, SPCA Selangor, Vegan Outreach, Veganuary, VegeProject Japan, We Animals Media, and World Animal Protection.

[5] We would have categorised all three of these individuals as leadership anyway.

[6] Unfortunately, due to the small number of respondents in this category, we had to exclude results of this subgroup analysis from the subgroup analyses spreadsheet to protect the respondents’ anonymity.

[7] This included legal actions, investigations, capacity-building programmes, journalism, and any group with a substantial focus on animals other than farmed animals.

[8] Where we identified financial reports or reviews from Animal Charity Evaluators with clear breakdowns of spending by programme type, we included an organisation in a category if it spent 10% of its budget or more on a particular intervention type. If we did not find this sort of financial data, we relied on descriptions on the organisation’s website.

[9] Note, the global North vs. global South divide is a socio-economic and political categorisation, not simply a geographic one. We followed this map for our categorisations.

[10] Unfortunately, the change in recommendations after we sent out the surveys meant that some organisations that are now recommended were not invited to participate.

[11] We gave the guidance: “E.g. you might count staff who work 20 hours per week as 0.5 each. Please include all current employees and independent contractors, regardless of the length of their contracts.”

[12] We asked the question “How many full-time equivalent, paid staff are there in your organisation?” and gave the guidance: “E.g. you might count staff who work 20 hours per week as 0.5 each. Please include all current employees and independent contractors, regardless of the length of their contracts.”

Participants’ responses to a particular question were multiplied by their response to the question about the number of FTE paid staff. This number was then divided by the sum of responses to the question about FTE paid staff across all organisations which participated in the question of interest.

Most surveyed organisations focus exclusively on farmed animals. However, this was not the case for some of the largest organisations surveyed. To prevent their answers dominating in the weighting system and to reflect the fact that we are most interested in the farmed animal movement, we counted these organisations as having only 51 full-time paid employees in the weighting. This number was chosen fairly arbitrarily (representing the boundary of the medium and large size categories) as we do not know the number of staff working specifically on farmed animal issues. Making this change had quite a substantial effect on some results.

For the one organisation that did not provide an answer about the number of full-time equivalent, paid staff that they had, we inputted an answer from a publicly available document which specified the number of staff they had in 2017. This number is presumably not accurate, but may still be roughly correct; using this number enabled us to include them in the weighted analyses.

[13] In weighting 2, current ACE “Top charities” were given a multiplier of 1.5, “Standout” charities a multiplier of 1.25, former ACE top or standout charities a multiplier of 1 and organisations that had not yet been recommended by ACE a multiplier of 0.75. This multiplier was then applied to their score from weighting 1, i.e. their response to a particular question multiplied by their response to the question about the number of FTE paid staff. This number was then divided by the sum of (responses to the question about FTE paid staff multiplied by their ACE recommendation multiplier) across all organisations which participated in the question of interest.

Weighting 3 was the same except that the multipliers used were 10, 5, 2, and 1, rather than 1.5, 1.25, 1, and 0.75.

[14] Across the 33 numerical scores for which we calculated an average score, the average difference between the unweighted score and weighting 1 score was 0.11. (For the purpose of this calculation, regardless of whether the unweighted or weighting 1 score was higher, the difference that was included in the average was a positive number.) The average difference between the unweighted score and weighting 2 score was 0.14. The average difference between the unweighted score and weighting 3 score was 0.25. (The differences in this footnote were calculated before we added in the responses for one late respondent.)

[15] For two of the organisations, we invited two individuals with very different roles. All four of these “duplicate” invited individuals provided responses. Two individuals from another organisation also both provided responses. The meta respondents that gave us permission to share their organisation’s name were: Animal Charity Evaluators, Animal Ethics, Charity Entrepreneurship, Faunalytics, The Greenbaum Foundation, The Humane League (Open Wing Alliance), Open Philanthropy, Rethink Priorities, Sentience Institute, and World Animal Net.

[16] We added weighting 3 as a robustness check after seeing how little difference there was between weighting 2 and the unweighted results. We added a subgroup analysis for Asian organisations compared to non Asian organisations after realising that several Asian organisations could be classified as being part of the global North, despite perhaps being in a very different context to other organisations in the global North. We added one late respondent to the analysis after seeing the initial results (which made little difference to the results).

[17] You might be inclined to take asterisked limitations more seriously; these are less likely to be spurious criticisms that we invented after seeing surprising results in order to justify our prior beliefs.

[18] We added that “this includes nonprofits (including but not limited to those recommended by Animal Charity Evaluators), companies producing or selling alternative foods to animal products, plus individuals pursuing careers in other areas (e.g. politics and policy, academia, animal law, or earning to give) and aiming to maximise their positive impact for animals.”

[19] Changed wordings for meta respondents were: “Unfavourable attitudes among targets of advocacy, e.g. companies, government, consumers),” “Lack of public awareness of the work of specific EAA organisations,” “Difficulties coordinating between EAA organisations and stakeholders,” and “Difficulties coordinating internally within specific EAA organisations.”

[20] This is perhaps unsurprising given the substantial support for corporate welfare campaigns from Open Philanthropy Project, Animal Charity Evaluators, and The Humane League’s Open Wing Alliance.

[21] Researchers have tended to agree that institutional tactics should be prioritised over individual tactics. This has been reflected in the funding from groups like Open Philanthropy and Animal Charity Evaluators.

[22] The range of average scores was wider for the South (1.8 to 3.8 rather than 1.9 to 3.3), though this could just be due to the smaller sample size meaning that outliers have a larger effect on the average score.

[23] After being asked, “(Optional) Is there anything you’d like to add? E.g. other factors that limit the EAA community’s efficiency or impact, elaboration on any factors you marked as a 4 or 5,” one respondent suggested each of the following factors:

• “High rates of turnover in EAA staff that is not yet well-understood” (5)
• “The presence of racism within the EA community” (5)
• “Rivalry from other local nonprofits in the sector” (5)
• “Lack of collaboration with other progressive social movements” (4)
• “Lack of team capacity” (4)
• “An overly dogmatic understanding of ‘objective’ effectiveness” that overvalues behavioural and direct outcomes at the expense of attitudinal and indirect outcomes (3)
• “Poor hiring processes” (3)
• “Poorly considered big picture strategy” (3).

[24] Exceptions might be if you think that the targets of advocacy just need behavioural support to actually act on their values or if the targets of advocacy are very specific groups of individuals whose actions are constrained by other stakeholders.

[25] The logic of the two options offered in the question wording — “working for 10 years building up experience and expertise” vs. “50% of the salary of that role” — was that these both seem like goals that many individuals in the global North could realistically achieve if they strove for it over a long period of time. Our “Effective Animal Advocacy Nonprofit Roles Spot-Check” identified an average salary among advertised roles of $42,000, or $50,600 if roles from India, Mexico, Brazil, and France are removed. Individuals in well-paid roles in the corporate sector might well be able to donate ~$25,000 a year if they focused their career on earning as much money as possible.

Additionally, we were keen to keep the question as easily comprehensible as possible to reduce difficulties of interpretation of the results, so the round numbers of 10 years and 50% were convenient.

[26] As shown in the section below, “Campaigns, corporate engagement, or volunteer management” had a moderate rating relative to other role types, in terms of difficulty in hiring and retaining high-quality staff.

[27] See also 80,000 Hours’ articles on the highest paying jobs and on earning to give more generally.

[28] We provided the following guidance: “Of course, there are many factors that could substantially influence this answer, but please try to answer just assuming your best guess or the “average” for those other relevant factors, rather than the best or worst case scenario. Please give your answer in US dollars.”

[29] One direct work respondent also commented (without having seen the wording of the question given to meta respondents) that “top talent is a lot more valuable than money for most hard-to-fill roles… until the person is giving at least $250,000/year, IMO.”

[30] Though using weighting on this version brings “Natural sciences” scores back up to a similar amount.

[31] This was more so for the role type that they were less specialised in, i.e. “Corporate/producer welfare campaigns” organisations struggled more with “Government, policy, lobbying, or legal” and “Political campaigns or engagement” organisations struggled more with “Campaigns, corporate engagement, or volunteer management.” Both also struggled a little more with “Research” roles.

[32] To a lesser extent, the same was true for “Campaigns, corporate engagement, or volunteer management” roles.

[33] This matches up with the higher ratings for limits in effectiveness and efficiency caused by “Lack of (qualified and capable) activists and volunteers” and “Lack of (qualified and capable) applicants for paid roles” for organisations working primarily in Asia (4.2 and 3.9) than elsewhere (2.1 and 2.6).

[34] For example, one respondent noted that they were “unsure” about the second part due to the newness of their organisation, but filled in every answer with “3” (in this case, we manually removed the responses to count them as non-responses, though others may have done this and just not told us). One respondent noted that if something didn’t apply to them, they filled in “1”. One of the respondents that we trialled the survey with verbally (before having added the guidance about skipping items) seemed to answer such questions based on their impression of the experiences of other organisations.

[35] The relevant comments were:

• “Sophisticated legal counsel are difficult to find. There are a lot of passionate lawyers out there who do not have sufficient aptitude or experience to manage legal functions for a global organization.” (This respondent only put 2 for hiring “Government, policy, lobbying, or legal”.)
• “Financing/accounting has been very challenging for us.” (This respondent only put 3 for hiring “Operations, administration, or HR”.)
• “Under the ‘hiring’ question, I would break out campaigns from corporate engagement. I would mark hiring campaigns roles as a ‘2’, and would mark ‘corporate engagement’ roles as a ‘5’... Finding mission-aligned people who are qualified and have the gravitas to sit at the table with corporate executives, and will also work within our salary bands” is very difficult.” (This respondent put 3 for hiring “Campaigns, corporate engagement, or volunteer management”.)

[36] The relevant comments were:

• “Particularly challenging to identify campaign, corporate engagement etc candidates in countries with cultures that are less accepting of advocacy work eg China, Thailand, and some other countries.” (This respondent only put 2 for hiring “Campaigns, corporate engagement, or volunteer management”.)
• Similarly, one respondent noted that it is “far harder to find a lobbyist or advocate in a location where NGO pressure on government reform is not as accepted” (This respondent only put 2 for hiring each of “Government, policy, lobbying, or legal” and “Campaigns, corporate engagement, or volunteer management”.)

[37] The evidence is that:

• The average rating for the difficulty in hiring candidates of particular categories of roles or types of expertise was higher (2.9) than the average rating in retaining candidates of particular categories of roles or types of expertise (2.2).
• Three respondents explicitly noted that retention was not much of a problem for them or was less of a problem than hiring. There were also slightly more miscellaneous comments about hiring difficulties than about retention difficulties.
• The range of average scores across categories of roles or types of expertise was about twice as wide for hiring (2.5 to 3.6) as for retaining (2.0 to 2.6). This provides some evidence that, to the extent that talent bottlenecks arise, they are more likely to be caused by difficulties in hiring than difficulties in retention, though this might not necessarily be true (e.g. if you think that the differences are smaller but that these small differences still matter more). This also provides some evidence that interventions that affect retention may work similarly well across different categories of roles.

[38] We noted at the top of the section: “IMPORTANT: For the questions in this section, imagine the next 100 highly motivated, highly competent individuals looking for careers where they can have a positive impact for animals. Imagine also that they are all completely flexible about which sorts of roles that they apply for, but that once they pick a career path, they are unable to switch out of it for at least 5 years. Imagine also that you have complete control over where to allocate their efforts. In your answers to the questions below, please ensure that your answers add up to 100%. I.e. each category should have a number between 0 and 100, with an average of 12.5.” We then asked “What percentage would you get to focus their careers on: X?” eight times, replacing X with each of the sectors in the table below.

One respondent provided answers that added up to 97%, rather than 100%; since the numbers are close, we decided to include this respondent anyway. One respondent answered one option with 90% and another with 10%; they left the others blank, but we manually filled these in as 0, since this appeared to be their intention.

[39] Specifically, four individuals gave the examples of overlap of “Related legal work” with other categories: “Nonprofits” (twice), “Politics and policy,” and “Academia.” One respondent also noted possible overlap between “Nonprofits” and “Politics and policy”; in this case, our intention had been for the latter category to represent roles in public institutions rather than nonprofits, though this was not clear.

[40] One respondent noted this and commented: “I would want many more people in China to work on starting a for-profit alternatives company, many more people in SE Asia working in policy, many more people in the US working in existing organizations.”

[41] The respondent noted that they would have redistributed half of their “Working at existing nonprofits” responses into “Setting up new nonprofits” if they were responding for individuals late in their career.

[42] For example, you might think that those with high competence and motivation should most often focus on working directly in nonprofits (or animal product alternatives companies), but that other individuals should most often focus on earning to give. Large numbers of such individuals could increase the funding pool of the movement, but they would be unlikely to be better than other candidates for direct work. I.e. earning to give might be their comparative advantage even if they do not have outstanding personal fit with it.

[43] See footnote 12.

[44] We provided the guidance: “If needed, multiply hourly rates by 2,000 to convert to the equivalent full-time, annual salary.”

[45] We provided the guidance: “Include the time of other staff involved in the process, but not the salary of the newly hired worker. Exclude onboarding costs.”

[46] We provided the guidance: “Include any salary costs from staff time (delivery and participation), operational costs, and payments to external providers.” In this figure, we have excluded one suspected error.

[47] Our “Effective Animal Advocacy Nonprofit Roles Spot-Check” identified an average salary among advertised roles of $50,600, if roles from India, Mexico, Brazil, and France are removed. (France was removed because the relevant data came from one organisation which seemed to be an outlier.) This matches the figure in this survey for the global North quite closely.

[48] There were actually two outliers, both from organizations based in the global South. We contacted both organizations for clarification but only one responded; this organization corrected the figure given from $20,000 to $1,000.

[49] Here are the main identified examples from academic research:

• Blatter, Muehlemann, and Schenker (2012) found, in their research on Swiss companies, that average hiring costs were roughly $15,000 (10 to 17 weeks of wage payments). Of this, 29% was from “recruitment costs” (including costs for job postings, interviews, personnel costs, and external advisors) and the rest was from the “adaptation costs” (80 day average adaptation period, decline in productivity, direct training costs). The shares between these two categories of costs didn’t differ much by sector or occupation.
• Muehlemann and Pfeifer (2016) found that, among German companies, recruitment costs (average just under $2,000 and 35% of the total) were lower than adaptation costs (~$3,650).
• Jones’ (2008) research on the costs of turnover among registered nurses (RNs) suggests that “advertising/recruiting” ($3,378) and “hiring” ($2,679) again make up a smaller proportion of the total costs than “orientation/training” ($6,333), “newly hired RN productivity” ($7,169 for new RNs and $1,195 for experienced RNs) and “pre-turnover productivity” ($2,629).
• Buchbinder et al.’s (1999) research on turnover costs for physicians again suggested that the total turnover costs ($236,383) were mostly explained by loss of productivity ($196,333, i.e. 83% of the total). This included “specific training costs of a new primary care physician, including, but not limited to, orientation costs, increased time seeing patients, and increased utilization of laboratory tests and radiologic procedures.”
• Blake (2006) summarised that, “SHRM, the Society for Human Resource Management, estimated that it costs $3,500.00 to replace one $8.00 per hour employee when all costs — recruiting, interviewing, hiring, training, reduced productivity, et cetera, were considered. SHRM’s estimate was the lowest of 17 nationally respected companies who calculate this cost! Other sources provide these estimates: It costs you 30-50% of the annual salary of entry-level employees, 150% of middle level employees, and up to 400% for specialized, high level employees!”

[50] If we focused only on respondents from “large” organisations in our survey (51 staff or more) and “small” organisations in the magazine’s survey (100 to 999 employees), then the figures are $3,288 and $1,511, respectively, suggesting that animal advocacy nonprofits might substantially outspend other organisations on staff training.

[51] Indeed, if we adjusted for this bias, then the responses of the meta and direct work organisations might look very similar, with the exception of much lower support for “Mentorships between staff in industry/government and effective animal advocacy organisations” among meta respondents and somewhat higher support for “Mentorships between people with more experience and less experience in the effective animal advocacy movement.”

[52] The reported wording is from the meta survey. The respondents in the direct work survey were asked about their organisation specifically: “If your organisation could have all-expenses paid, high-quality training for all the staff members whose job descriptions refer primarily to tasks in one of the following categories, which type of training would you be most excited about?”

[53] However, we did not previously expect that respondents would be biased in this direction, because you might expect that they would also have an inflated sense of their own talents and therefore not believe that they would need training on management and leadership. In fact, one of us expected that these respondents would be biased towards selecting “Middle or junior managers” over “Leadership or senior managers” for this reason.

[54] Though we have low confidence in these views, concerns of our’s include:

• Overrepresentation of corporate welfare campaigns.
• Underrepresentation of organisations using less common institutional strategies, such as building grassroots political movements or litigating on behalf of animals.
• Overrepresentation of organisations based in the global North.