Initial research on social movements & protest

By JamesÖz, Sam Glover @ 2022-03-28T14:02 (+39)

Who we are (in a nutshell)

Social Change Lab is a new EA research project that is conducting research into whether protest could be a cost-effective way to achieve positive social change. We’re initially focusing on climate change and animal advocacy, and trying to understand whether protest should play a larger, or smaller, role in accelerating progress in these cause areas. Whilst more speculative, we’re also considering if protest can play a role for other issues, such as pandemic prevention, immigration reform, and existential risks.

 We put out some preliminary research in November last year, looking at the role that Extinction Rebellion UK played in making climate change more salient in the UK, and giving an overview of a small amount of the literature on the efficacy of protest. At the moment, we’ve just finished two literature reviews: one on the impact of protests and Social Movement Organisations (SMOs), and one on factors that affect success rates for protests. We are grateful to have received funding from the EA Infrastructure Fund to cover our expenses until July 2022.

Context

Whilst EA has the goal of doing as much good as possible, it’s not clear what the best ways to do this yet are. In a survey of 40 EA community leaders, people were most excited about greater “EA exploration” i.e. greater research into possible causes, interventions and ideas that could allow us to do more good. In particular, surveyed community leaders were interested in bringing in ideas from outside of traditional EA discourse. Similarly, as noted by Charity Entrepreneurship, there is lots of opportunity for organisations that engage in Exploratory Altruism, and we intend to explore the possibility that protest could be more cost-effective than other EA-recommended routes to change.  

Generally, as a community, we’ve been pretty research-focused in trying to understand the best ways to improve the world. However, much of this work hasn’t touched on how social movements interact with the world, and how they affect social change. ^[1] We see this as an opportunity to add value by providing research that addresses some unanswered questions about the role of social movements in improving the world.

Specifically, we think this work can be valuable to two audiences in particular:

1. Philanthropists seeking to fund the most effective routes to positive change in their respective cause area, where SMOs utilising protest could be a contender. For example, if we discover that animal advocacy protests are more cost-effective in reducing animal suffering relative to existing funded work, it makes sense we also fund these opportunities (all else being equal) if we seek to maximise our impact.
2. Social movements who utilise protest currently, who can employ and integrate best practices from social science literature to make their campaigns more effective.

Why Protest?

We believe that there is moderate evidence that protest can have an impact on public opinion, voting behaviour, public discourse, legislator behaviour, and corporate behaviour. Our interest in protest began by examining the impact of Extinction Rebellion, whose protest activity coincided with an increase in concern around climate change in the UK - as can be seen in the figure above. The concern about climate change in Europe and the United States has been increasing rapidly, and we believe that it is plausible that this increase has been at least partially due to protest activity by Extinction Rebellion, Fridays for Future, and Insulate Britain. That being said, it is clearly not the case that the increased salience of climate change is sufficient evidence to demonstrate that there is a causal connection between protest activity and concern about climate change. We think that if the claimed impacts of protests are accurate, that they can significantly alter public opinion or affect policymaker’s beliefs, then there is a strong case that EA should be considering this type of advocacy amongst others, such as direct policy advocacy. 

Our Research

Our Progress So Far

Some preliminary research we conducted last year suggested that Extinction Rebellion may have abated 16 tons of GHGs per pound spent on advocacy, making it 12x more effective than the Clean Air Task Force (CATF), the top EA recommended climate change charity. Whilst this is far from conclusive, with plenty of valid critiques raised of this research, we think it’s enough evidence to look into this topic with greater detail.

Currently, we are finishing off several initial projects, whose results can be seen below, namely: a literature review on protest outcomes, a literature review on factors that are likely to make protests successful in achieving their aims and a series of interviews with grantmakers to understand their current beliefs and uncertainties around protest. For our literature reviews we are focusing on academic papers that use quasi-experimental techniques and experiments, as well as looking at papers that examine observational data while also having rigorous causal identification strategies. 

Literature Review on protest outcomes

We have found research that leads us to believe it is likely that there is a causal impact of non-violent protest in positively affecting public opinion, voting behaviour, media coverage and policy. The summary of our literature review that highlights some of these points is as below, where you can see the full literature review here. A database of relevant research papers we’ve included in this research can be seen here. It’s important to note that this is the first draft of our literature reviews, pre-feedback from relevant experts, so it’s likely they’ll be refined and improved going forward. Feedback and comments are very much welcome, on this post or via email.

• Whilst positive effects of protest on public opinion, public discourse and voting behaviour have moderate evidence supporting each outcome, effects of protest on policymaking and policymakers are more mixed.^[2] Specifically, impacts of protest on policy seems highly context dependent, on factors such as existing political structures and current public opinion.
• There is some debate whether protest influences political attitudes, or it simply amplifies existing public preferences, with various studies lending support to both arguments.
• The effect sizes of protest on public opinion and voting behaviour are somewhat small, yet quite significant in the realm of politics, with shifts of approximately 2-5% found via natural experiments. In the studies we examined, there were noticeable impacts on electoral outcomes as a result of protest activity. In experimental conditions, effect sizes have been found to be both null and larger than 5%, depending on the study.
• There is strong evidence that protest can be effective in North America and Western Europe, specifically within issues of civil rights, climate change, and social welfare. For countries in the Global South, there is very little research into protest outcomes, so generalising these findings to other regions is quite tenuous.
• We think that the evidence for short-term and medium-term change is much stronger than the evidence for long-term change. This is largely because research designs that are able to make causal inferences are almost necessarily short-term - research using experiments or quasi-experimental designs largely examine short-term or medium-term effects. There is currently very little literature on the long-term impacts of protest on public opinion or public discourse.

Summary Table

Note: Confidence ratings are based on the number of available studies supporting the claim. Low = 0-2 studies supporting, or mixed evidence; Medium = 3-6 studies supporting; High = 7+ studies supporting.

This initial review of the literature suggests to us that it is highly plausible that protest is an effective strategy in some cases, and that it is worth exploring the possibility that protest is more cost-effective than current EA recommendations. Again, we encourage interested folks to read the full literature review on protest outcomes here.
 

Literature Review on protest movement success factors

• We believe that there is moderate evidence that the protesters are more likely to succeed if they are perceived as a diverse group, if they have a unified message, and if the number of protesters is high.
• The factors that influence the public are slightly different to factors that influence policymakers. Namely, the public is much more concerned about the worthiness of protestors, whilst policymakers are more influenced by the numbers of protestors and the diversity of groups present.
• We believe that there is strong evidence that non-violent protest is preferable to violent protest for achieving desired outcomes. There have been multiple studies that indicate that peaceful protesters are more likely to persuade the public - including Wasow (2020) and Feinberg et al. (2017).
• The political context in which a protest takes place is also relevant: there is some evidence that protesters are more likely to achieve their aims if they highlight an issue about which the public is already on their side, and that they are more likely to have an influence at the early stages of the legislative process rather than the later stages. We believe in some cases, the political context might be important enough to dominate over factors within the movement’s control, whereby detrimental external conditions might lead to failure regardless of how well the movement crafts its strategies or tactics.
• There is generally not much literature on this topic, with many possible areas for future study to understand best practices for social movements and protest.

You can see our full literature review on protest movement success factors here.

Grantmaker Interviews

We interviewed grantmakers from 8 different grantmaking institutions within the EA community (e.g. EA Funds, Open Philanthropy, Founders Pledge, etc.), working across animal welfare, climate change and building the EA movement. In addition, we interviewed one partially EA-aligned animal welfare grantmaker, and one climate grantmaker who funds a significant proportion of existing climate protest, but these results are not included in the tables below. These interviews ranged from 30-60 minutes, with the purpose to ascertain the existing uncertainties and beliefs grantmakers had around protest as an intervention.

The results can be seen in the tables below. We asked many more questions, but the most relevant and important questions can be seen below. Generally, we were trying to understand the prior beliefs of grantmakers, so asked fairly open-ended questions. We intended to release more full summaries of our interviews closer to the final publication of our report, in several months.

Other research methods we asked about included natural experiments (1 person for), public opinion polling (2 people for), Twitter research (2 against). As you might be able to see from our table above, there was a fair bit of disagreement amongst grantmakers on what kinds of research they would find the most useful. This doesn’t make our life easy when trying to prioritise our future research methods sadly! There were some patterns we noticed e.g. animal-focused grantmakers preferred case studies over grantmakers from other cause areas, due to a lack of focus on animal advocacy in the existing protest literature.

Informal EA Survey

We also conducted a small informal EA community survey (with 49 respondents) who ranked on a 1-5 Likert Scale how compelling they found the above research methods. This was done mainly out of interest and informed our decision-making to a very small degree, but the results can be seen below for reference. Interestingly, the EA community (that we sampled) and grantmakers we interviewed had somewhat different priorities, with policymakers interviews being the exception that both groups were excited about. However, we wouldn’t put too much weight on this survey as we did end up adding more research methods after this survey was conducted, based on further conversations.

Next Steps

We have several ideas for research over the next few months, based on interviews with grantmakers and other researchers, and on the responses we got to our survey of members of the EA Community.

• We intend to continually improve both our literature reviews, based on findings from our expert elicitation (see below) as well as feedback from the EA community and various reviewers.
• We are in the early stages of undertaking an expert elicitation, where we are talking to academics who research protest. This project has two aims: 1) Better understand existing protest literature by interviewing relevant academics in the field and 2) Attempt to elicit answers to key open questions which are extremely challenging to answer otherwise. Examples of these questions can be seen below in the “Key Questions” section.
  • On methodology: We intend to conduct interviews with 10-20 academics, which we hope will provide some insight to EAs and grantmakers about the current degree of academic consensus on certain complex yet crucial questions. We will publish summaries of our interviews that will help people understand which questions are settled, and which remain contentious among experts. We aim to do this on both the topic of protest outcomes, as well as which factors lead to successful protest movements.
• We are planning to conduct our own public opinion polls before and after upcoming climate protests in the UK. We hope this will shed some light on the impacts of protest on public opinion using a more controlled method that has previously not been used in the literature. We are working with academics and YouGov to build and analyse this survey.
• We intend to publish articles in mainstream media to try and get some of our research out to non-EA audiences. This will be contingent on opportunities that arise - depending on how much media coverage protests in the next few months receive, we may be able to get op-eds published in a mainstream UK newspaper. We also intend to publish articles on how protest can be used to benefit people in specific cause areas - for instance, we are currently drafting a piece for Faunalytics about how animal advocates might be able to make use of protest as a strategy.
• We want to talk with SMOs directly about their bottlenecks, and will survey movement leaders to understand what protest movements would do with additional funding, as well as understanding their general limitations to more impact.
• We are looking into conducting more Cost-Effective Analyses (and case studies) of modern protest movements, with specific interest in animal advocacy movements.

Key Questions we’ve identified for further research

After talking to grantmakers and hearing from other EAs, some concerns and questions have been raised repeatedly that we are keen to think about:

• To what extent is there likely to be a backfire effect of protest? Can protest sometimes cause people to become less sympathetic to the cause of the protesters? Are there ways of preventing this? What are the reputational risks for funders of protest movements?
• How generalisable is most of the research on protests? For instance, can we really know much about how successful protests for animal welfare are likely to be based upon research on climate protest? How does the political context of different countries affect the potential efficacy of social movements? How much can we really learn from a few quasi-experimental studies and experiments that use news vignettes?
• Are SMOs really bottlenecked by funding, or are there other bottlenecks that are more important for protesters?
• Which tactics, strategies and factors of protest movements (e.g. size, frequency, target, etc.) most affect chances of success? And how important is movement agency relative to external political conditions?
• Should protesters demand a specific policy change or is it more effective to just highlight a broader cause area?
• Can protest exacerbate polarisation? What role does polarisation play in bringing about legislative change? Could it be good to polarise people on a certain issue in some instances?
• To what degree are the impacts of protest mediated via the media, public opinion, or other avenues? Which theory of change pathways have the strongest evidence?

Challenges

There are several reasons why we might not be successful in our aims to answer some of these thorny and complicated questions around the impact of protest on social change:

• We are only a small organisation with two full-time employees, and with our current funding we only have until July to carry out our research. It’s highly unlikely we’ll get conclusive evidence in this timeframe that will update us strongly in one direction or another. Instead, we are trying to narrow the uncertainty around protest as an intervention, to understand if it’s worth greater time or financial investment.
• This information is hard to gather, the results are hard to measure and datasets on protest movements are either hard to obtain or simply do not exist. Unlike some EA interventions, it is nearly impossible to conduct randomised controlled trials (RCTs) on the impact of protest. This makes the question “Are protests a cost-effective intervention relative to other interventions?” very challenging to answer conclusively. Often, the impacts of protest are mediated through other mediums (e.g. the media), happen over long time periods or are said to have broad cultural impacts, which are challenging or impossible to quantify accurately.
• In addition, the world is extremely complex and messy, so it’s unsure how much any previous literature or findings on the impact of protest will generalise to other contexts, whether that’s other countries, issues, or time periods.
• It is also worth noting that neither of us are academics, and so getting to grips with all of the literature and trying to figure out which papers are the best will take us longer than it would take people who were already familiar with the state of the literature on protest and SMOs

Who we are

James Ozden has spent the last three years working within social movements to tackle climate change and reduce animal suffering, as the Director of Animal Rebellion. In addition, he worked on strategy for one of the most well-known social movement organisations in recent history, Extinction Rebellion. More recently, James completed the Charity Entrepreneurship Incubation Program on how to launch highly impactful nonprofits using EA principles. 

Sam Glover recently graduated from a Master’s degree in Democracy and Comparative Politics at UCL - his Master’s thesis involved using machine learning on Twitter data to establish how legislators react to Eurosceptic pressure in their constituency, and helped familiarise him with advanced quantitative methods. Sam recently joined Social Change Lab as a Research Manager, where he helps carry out our research and literature reviews on the effects of protest and social movements.
 

How You Can Help

Feedback

It would be hugely beneficial for us to receive feedback from the members of the EA Community. While we have already put out a survey to try and get a better understanding of what kind of research EAs would find beneficial (which you can fill out here), it would also be really helpful to hear specific and detailed feedback from EAs who have views on research around the efficacy of protest. If you’re interested in giving feedback on our literature reviews, surveys or other work, please contact us using our emails below!

Similarly, if you are someone with an understanding of how effective protest has been for a specific cause area, we would be keen to talk to you. This could be for the two main cause areas we’re focusing on, animal advocacy and climate change, or other areas that we haven’t yet explored.

Funding

We are currently operating on a grant from the EA Infrastructure Fund which will last until June 2022. We have been talking to grantmakers and advocates to get a better understanding of how we can benefit the EA community, and will be conducting some greater impact evaluation of our work towards the end of our project. If we and others believe our work is impactful and useful, we will be seeking funding to continue our operations past June. If our work is not as helpful as we hoped, we don’t plan on continuing.

Further Information

Website: https://www.socialchangelab.org

LinkedIn: https://www.linkedin.com/company/social-change-lab/

Contact James: james@socialchangelab.org

Contact Sam: sam@socialchangelab.org

1. ^{^}

   There are obvious exceptions, with a good compilation of social movement work done by EAs here.

2. ^{^}

   By moderate evidence, we generally mean 3-6 studies supporting a claim. By strong evidence, we mean 7 or more studies supporting a claim. This is somewhat arbitrary, and our lines could be drawn in different places.