Initial research on social movements & protest

By JamesÖz, Sam Glover @ 2022-03-28T14:02 (+39)

Who we are (in a nutshell)

Social Change Lab is a new EA research project that is conducting research into whether protest could be a cost-effective way to achieve positive social change. We’re initially focusing on climate change and animal advocacy, and trying to understand whether protest should play a larger, or smaller, role in accelerating progress in these cause areas. Whilst more speculative, we’re also considering if protest can play a role for other issues, such as pandemic prevention, immigration reform, and existential risks.

 We put out some preliminary research in November last year, looking at the role that Extinction Rebellion UK played in making climate change more salient in the UK, and giving an overview of a small amount of the literature on the efficacy of protest. At the moment, we’ve just finished two literature reviews: one on the impact of protests and Social Movement Organisations (SMOs), and one on factors that affect success rates for protests. We are grateful to have received funding from the EA Infrastructure Fund to cover our expenses until July 2022.

Context

Whilst EA has the goal of doing as much good as possible, it’s not clear what the best ways to do this yet are. In a survey of 40 EA community leaders, people were most excited about greater “EA exploration” i.e. greater research into possible causes, interventions and ideas that could allow us to do more good. In particular, surveyed community leaders were interested in bringing in ideas from outside of traditional EA discourse. Similarly, as noted by Charity Entrepreneurship, there is lots of opportunity for organisations that engage in Exploratory Altruism, and we intend to explore the possibility that protest could be more cost-effective than other EA-recommended routes to change.  

Generally, as a community, we’ve been pretty research-focused in trying to understand the best ways to improve the world. However, much of this work hasn’t touched on how social movements interact with the world, and how they affect social change. [1] We see this as an opportunity to add value by providing research that addresses some unanswered questions about the role of social movements in improving the world.

Specifically, we think this work can be valuable to two audiences in particular:

  1. Philanthropists seeking to fund the most effective routes to positive change in their respective cause area, where SMOs utilising protest could be a contender. For example, if we discover that animal advocacy protests are more cost-effective in reducing animal suffering relative to existing funded work, it makes sense we also fund these opportunities (all else being equal) if we seek to maximise our impact.
  2. Social movements who utilise protest currently, who can employ and integrate best practices from social science literature to make their campaigns more effective.

 

Why Protest?

We believe that there is moderate evidence that protest can have an impact on public opinion, voting behaviour, public discourse, legislator behaviour, and corporate behaviour. Our interest in protest began by examining the impact of Extinction Rebellion, whose protest activity coincided with an increase in concern around climate change in the UK - as can be seen in the figure above. The concern about climate change in Europe and the United States has been increasing rapidly, and we believe that it is plausible that this increase has been at least partially due to protest activity by Extinction Rebellion, Fridays for Future, and Insulate Britain. That being said, it is clearly not the case that the increased salience of climate change is sufficient evidence to demonstrate that there is a causal connection between protest activity and concern about climate change. We think that if the claimed impacts of protests are accurate, that they can significantly alter public opinion or affect policymaker’s beliefs, then there is a strong case that EA should be considering this type of advocacy amongst others, such as direct policy advocacy. 

 

Our Research

Our Progress So Far

Some preliminary research we conducted last year suggested that Extinction Rebellion may have abated 16 tons of GHGs per pound spent on advocacy, making it 12x more effective than the Clean Air Task Force (CATF), the top EA recommended climate change charity. Whilst this is far from conclusive, with plenty of valid critiques raised of this research, we think it’s enough evidence to look into this topic with greater detail.

Currently, we are finishing off several initial projects, whose results can be seen below, namely: a literature review on protest outcomes, a literature review on factors that are likely to make protests successful in achieving their aims and a series of interviews with grantmakers to understand their current beliefs and uncertainties around protest. For our literature reviews we are focusing on academic papers that use quasi-experimental techniques and experiments, as well as looking at papers that examine observational data while also having rigorous causal identification strategies. 

 

Literature Review on protest outcomes

We have found research that leads us to believe it is likely that there is a causal impact of non-violent protest in positively affecting public opinion, voting behaviour, media coverage and policy. The summary of our literature review that highlights some of these points is as below, where you can see the full literature review here. A database of relevant research papers we’ve included in this research can be seen here. It’s important to note that this is the first draft of our literature reviews, pre-feedback from relevant experts, so it’s likely they’ll be refined and improved going forward. Feedback and comments are very much welcome, on this post or via email.

Summary Table

FindingConfidence
Protest can have significant short-term impactsStrong
Protest can be effective in North America and Western EuropeStrong
Protest can have significant impacts on voting behaviour and electoral outcomesMedium
Protest can influence public opinionMedium
Protes can influence public discourse and media narrativesMedium
Protest can influence policyLow (mixed evidence)
Protest can influence policymaker beliefsLow (little evidence)
Protest can be effective in the Global SouthLow (little evidence)
Protest can have significant long-term impacts (on public opinion and public discourse)Low (little evidence)

Note: Confidence ratings are based on the number of available studies supporting the claim. Low = 0-2 studies supporting, or mixed evidence; Medium = 3-6 studies supporting; High = 7+ studies supporting.

 

This initial review of the literature suggests to us that it is highly plausible that protest is an effective strategy in some cases, and that it is worth exploring the possibility that protest is more cost-effective than current EA recommendations. Again, we encourage interested folks to read the full literature review on protest outcomes here.
 

Literature Review on protest movement success factors

You can see our full literature review on protest movement success factors here.


 

Grantmaker Interviews

We interviewed grantmakers from 8 different grantmaking institutions within the EA community (e.g. EA Funds, Open Philanthropy, Founders Pledge, etc.), working across animal welfare, climate change and building the EA movement. In addition, we interviewed one partially EA-aligned animal welfare grantmaker, and one climate grantmaker who funds a significant proportion of existing climate protest, but these results are not included in the tables below. These interviews ranged from 30-60 minutes, with the purpose to ascertain the existing uncertainties and beliefs grantmakers had around protest as an intervention.

The results can be seen in the tables below. We asked many more questions, but the most relevant and important questions can be seen below. Generally, we were trying to understand the prior beliefs of grantmakers, so asked fairly open-ended questions. We intended to release more full summaries of our interviews closer to the final publication of our report, in several months.

Other research methods we asked about included natural experiments (1 person for), public opinion polling (2 people for), Twitter research (2 against). As you might be able to see from our table above, there was a fair bit of disagreement amongst grantmakers on what kinds of research they would find the most useful. This doesn’t make our life easy when trying to prioritise our future research methods sadly! There were some patterns we noticed e.g. animal-focused grantmakers preferred case studies over grantmakers from other cause areas, due to a lack of focus on animal advocacy in the existing protest literature.


Informal EA Survey

We also conducted a small informal EA community survey (with 49 respondents) who ranked on a 1-5 Likert Scale how compelling they found the above research methods. This was done mainly out of interest and informed our decision-making to a very small degree, but the results can be seen below for reference. Interestingly, the EA community (that we sampled) and grantmakers we interviewed had somewhat different priorities, with policymakers interviews being the exception that both groups were excited about. However, we wouldn’t put too much weight on this survey as we did end up adding more research methods after this survey was conducted, based on further conversations.

Next Steps

We have several ideas for research over the next few months, based on interviews with grantmakers and other researchers, and on the responses we got to our survey of members of the EA Community.

 

Key Questions we’ve identified for further research

After talking to grantmakers and hearing from other EAs, some concerns and questions have been raised repeatedly that we are keen to think about:

 

Challenges

There are several reasons why we might not be successful in our aims to answer some of these thorny and complicated questions around the impact of protest on social change:

 

Who we are

James Ozden has spent the last three years working within social movements to tackle climate change and reduce animal suffering, as the Director of Animal Rebellion. In addition, he worked on strategy for one of the most well-known social movement organisations in recent history, Extinction Rebellion. More recently, James completed the Charity Entrepreneurship Incubation Program on how to launch highly impactful nonprofits using EA principles. 

Sam Glover recently graduated from a Master’s degree in Democracy and Comparative Politics at UCL - his Master’s thesis involved using machine learning on Twitter data to establish how legislators react to Eurosceptic pressure in their constituency, and helped familiarise him with advanced quantitative methods. Sam recently joined Social Change Lab as a Research Manager, where he helps carry out our research and literature reviews on the effects of protest and social movements.
 

How You Can Help

Feedback

It would be hugely beneficial for us to receive feedback from the members of the EA Community. While we have already put out a survey to try and get a better understanding of what kind of research EAs would find beneficial (which you can fill out here), it would also be really helpful to hear specific and detailed feedback from EAs who have views on research around the efficacy of protest. If you’re interested in giving feedback on our literature reviews, surveys or other work, please contact us using our emails below!

Similarly, if you are someone with an understanding of how effective protest has been for a specific cause area, we would be keen to talk to you. This could be for the two main cause areas we’re focusing on, animal advocacy and climate change, or other areas that we haven’t yet explored.

Funding

We are currently operating on a grant from the EA Infrastructure Fund which will last until June 2022. We have been talking to grantmakers and advocates to get a better understanding of how we can benefit the EA community, and will be conducting some greater impact evaluation of our work towards the end of our project. If we and others believe our work is impactful and useful, we will be seeking funding to continue our operations past June. If our work is not as helpful as we hoped, we don’t plan on continuing.

 

Further Information

Website: https://www.socialchangelab.org

LinkedIn: https://www.linkedin.com/company/social-change-lab/

Contact James: james@socialchangelab.org

Contact Sam: sam@socialchangelab.org

  1. ^

    There are obvious exceptions, with a good compilation of social movement work done by EAs here.

  2. ^

    By moderate evidence, we generally mean 3-6 studies supporting a claim. By strong evidence, we mean 7 or more studies supporting a claim. This is somewhat arbitrary, and our lines could be drawn in different places.


Dan Stein @ 2022-03-29T01:48 (+5)

Hi James, thanks for this and I feel like this research is super-helpful. As you know, we at Giving Green have also explored the question of protest (as a form of what we call "outsider legislative advocacy"), and are also generally bullish  on these techniques. But also, (as you mention), I think only a small minority of protest movements are really successful. We've had a lot of trouble identifying organizations we want to recommend in the context of climate policy in the US. 

We're looking forward to applying your findings as our search continues!

James Ozden @ 2022-03-29T10:55 (+2)

Hi Dan - thanks for this! Definitely agree in that protest movements can be hits-based and most don't do much but the best ones can be hugely influential. That's definitely one of the hardest questions to resolve e.g. how do we predict which movements will fall into the latter bucket a priori, hence our work on identifying factors of successful movements. We're planning on doing some more work on this in the next few months so will keep you posted and definitely hope it's helpful to Giving Green!

Emily Grundy @ 2022-03-29T00:30 (+4)

Really interesting topic, thanks for sharing James! I was wondering whether you could share any info about your lit review methods (e.g., how you found your included articles)?

James Ozden @ 2022-03-29T11:24 (+4)

Thanks Emily, much appreciated! I also really enjoyed your recent work on interventions that influence animal product consumption so thanks for doing that.

For methodology, that's a good point and definitely something we should include more information on so will do that for an updated version in the near future. Not sure if you saw it but we do have a database of resources we compiled whilst doing this if you want to see the inputs. 

On how we actually found the included pieces, this was a mix of methods, and we didn't do it in a systematic way akin to your work, although we might consider doing this in the future (suggestions welcome if you think is a good idea!). As we were mainly doing this for our own understanding and getting a lay of the land, we didn't think it was too crucial to do a systematic analysis (and our advisors also suggested this). But a few of the ways we did find papers:

  • Tools such as Google Scholar, ResearchRabbit and Elicit that helps find studies adjacent to your question or other studies you're interested in. We would use keyword searches such as "protest outcomes", "protest effectiveness", "impacts of protest", etc. for the outcomes, and similar variations of keywords for the success factors work. This is how we found the majority of the useful studies.
  • We looked at the research groups and prior publications of basically all the academics we found using the above method, which was especially useful to find newer papers and other academics who were newer in the field doing this work (e.g. just joined a relevant research group)
  • I read two academic-focused books on the relevant topics (How Social Movements Matter and Prisms of the People), Sam read 1-2 similar books, and we found literature via that
  • We interviewed 5 academics who had some influential papers in the various fields and asked them to recommend us the most important / key papers in the field which was useful to make sure we didn't miss anything crucial (we probably found 3-4 additional papers like  this).
  • Someone else had conducted a systematic analysis (sadly not public) on an adjacent sub-field within social movements so we found some useful papers this way too.

 

Quite roughly, I'll outline some of the criteria we used:

  • We only included studies that utilised protest for things other than regime change (e.g. we didn't include Erica Chenoweth's famous work on toppling dictators as this isn't really relevant to the types of protest we're interested in)
  • We didn't include studies from protests prior to the 1960s. Even though this boundary is slightly fuzzy, we think the political context from prior to this time was too different to current times to be useful.
  • We focused primarily on empirical papers rather than theory-based ones, although we did include a small amount of theoretical papers to explain the mechanism behind some of the findings we observed
  • We included study designs using observational and experimental methods
  • As there's only one meta-analyses on this topic (from the 1980s) we included mainly primary research papers and didn't have the option to reply on meta-analyses or systematic reviews.
  • In reality, there weren't that many papers that fit all our criteria as this is a reasonably small and under-studied field, so we think we covered the vast majority of papers that fit our criteria above
Ben Williamson @ 2022-03-29T08:46 (+3)

Really impressive work! I think the idea that protest movements could be very high-impact is pretty convincing. It's a lot of ground to cover to try and find more concrete answers to how much impact movements have and how this can be maximised but I think this report makes good progress towards that. 

More generally, great to see in-depth research on plausibly impactful topics that we've not looked at much as a community. 

James Ozden @ 2022-03-29T11:27 (+2)

Thanks Ben - this is much appreciated! Agreed there's still lots more work to be done to find these plausibly high-impact options, so fingers crossed we can make some decent headway. Likewise to you on Effective Self-Help, great to see your research on that front!