Nailing the basics – Theories of change

By Aidan Alexander, CE @ 2023-07-16T06:40 (+229)

Why write a post about theories of change?

As participants in a movement with ‘effective’ in its name, it’s easy to think of ourselves as being above falling for the most common mistakes made in the non-profit sector more broadly. We’re grappling with the cutting-edge stuff, not the basics, right?

But we need to beware of this kind of complacency. It’s crucial to nail the basics, like theories of change, and far fewer EA organizations do this than you’d expect. That includes Charity Entrepreneurship, the organization I work at, where we teach participants in our Incubation Program and Foundation Program about the importance of theories of change, and yet until today we didn’t even have our own theory of change on our website!

That’s why we decided to share this post on theories of change – potentially the first of a series on ‘nailing the basics’ of effective non-profit work, if people are interested. This post is mostly made up of an excerpt from our book on effective grantmaking (‘How to Launch a High-Impact Foundation’), followed by a discussion of the theory of change for our Incubation Program. 

TL;DR

• It is very common for non-profit organizations (even EA orgs) to have no public theory of change, or to have a poor one. This is a big problem, because theories of change are the business models of the nonprofit world – you shouldn’t launch, fund or work for a project that doesn’t have a strong one! 
• A theory of change explicitly articulates the cause-and-effect steps for how a project or organization can turn inputs into a desired impact on the world (i.e. it’s their theory of how they’ll make a change). They generally include the following sections:
  • Inputs / activities: What the project or organization does to create change (e.g. “distribute bednets”)
  • Outputs: The tangible effects generated by the inputs (e.g. “beneficiaries have access to malaria nets”)
  • Intermediate outcomes: The outputs’ effects, including benefits for the beneficiary, (e.g. “malaria nets are used” and "reduced incidence of malaria")
  • Impact: What we’re ultimately solving, and why the intermediate outcomes matter (e.g. “lives saved”)
• Best practices when crafting a theory of change (i.e. for creators):
  • Invest sufficiently in understanding the problem context (i.e. understanding the needs and incentives of the beneficiaries and other stakeholders, as well as barriers to change and the economic & political context)
  • Map the causal pathway backwards from impact to activities
  • Question every causal step (is it clear why A should cause B? how might it fail?)
• Hallmarks of an excellent theory of change (i.e. for reviewers):
  • A focused suite of activities
  • The evidence and assumptions behind each step are explicitly named
  • The relative confidence of each step is clear
  • It is clear who the actor is in each step
• Common mistakes to avoid in theories of change are:
  • Not making fundamental impact the goal (e.g., stopping at ‘increased immunizations’ instead of ‘improved health’)
  • Being insufficiently detailed: (a) making large leaps between each step, (b) combining multiple major outcomes into one step (e.g. ‘government introduces and enforces regulation’).
  • Setting and forgetting (instead of regularly iterating on it)
  • Not building your theory of change into a measurement plan

The ‘what’ and ‘why’ of a theory of change

Building something complicated without an explicit plan is risky. From skyscrapers to software, ambitious projects need detailed blueprints. When building an effective nonprofit organization, the theory of change is that blueprint.

In the same way that you wouldn’t hire a builder who intends to build you a home without a blueprint, you shouldn’t fund a leader who intends to build an organization or launch a project without a theory of change. If the builder planned to create something particularly cutting-edge, like a home made entirely out of recycled cardboard that doesn’t need heating or cooling, you’d be particularly adamant that the builder explains exactly how this is possible and what evidence or assumptions it depends on. The same applies when it comes to funding an ambitious project that aims to improve the world. 

A theory of change explicitly articulates the logical steps of a plan to achieve a specific goal. It is generally represented in the form of a cause-and-effect diagram. A well-designed theory of change allows an organization to clearly communicate its activities and how they lead to the desired outcomes, and then to impact.^[1]

In fact, theories of change are more important for charities than business models are for businesses, because businesses have better feedback loops. If a for-profit is based on a bad idea, or is badly executed, it will see poor revenue and it will soon go out of business. On the other hand, if a charity is ineffective, it may limp along for years without having any impact, squandering limited funding and talent in the process.

Foundations deciding which opportunities to fund should expect a clear theory of change as table-stakes for potential grantees. In addition to spelling out how the potential grantee intends to convert your funding into impact, theories of change will allow you to determine how confident you can be that impact will be achieved. Is the causal chain tight, or convoluted? Are the steps well substantiated by evidence, or based on tenuous assumptions? Can any assumptions be easily tested? What is the crux that the project’s success depends on? A smart nonprofit will use its theory of change to identify its priorities for measurement and evaluation, and as the blueprint for its first cost-effectiveness analysis,^[2] which foundations can then use to inform funding decisions.

Real world examples

Example 1: A research organization

Imagine an organization that wants to use research to make a difference. Without an explicit theory of change, their implicit mental model of how they might make an impact may look something like this:

Review literature ⭢ Find insights that seem useful ⭢ Publish them

This is not a good theory of change. It doesn’t properly outline the goal, it doesn’t explain how the actions will lead to that goal, and it doesn’t explain how they will measure what they are doing. Presumably, the final goal is to impact living beings positively, not just publish papers. Even if the organization works very hard and puts out many widely cited papers, it’s possible that it will not make any difference. 

Compare this implicit model to an explicit theory of change published by a young charity after they went through one of Charity Entrepreneurship’s early incubation programs:

^[3]

On the left, you can see the charity's core research agenda at a glance (the inputs/activities). On the right, you can see some measurable metrics such as money moved, new projects, and hires made (intermediate outcomes) which lead to the final impact – happier lives.

This is definitely an improvement on the implicit mental model: 

• It connects activities all the way to impact (more complete)
• It gives a clearer depiction of what the activities actually look like (more concrete)
• It makes it easy to spot that “communicating to individuals and key organizations” is a crux of this process, which needs extra care and attention when it comes to execution (more action-guiding) 

Having developed this theory of change, they will be more likely to give adequate focus to ensuring that they don’t just publish research, but that the right people read it and change their actions (e.g. fund more impactful organizations, start more impactful projects, or measure impact in a more appropriate manner). 

That being said, this theory of change has significant room for improvement. In particular, the intermediate steps between communicating the research and organizations changing their actions ought to be spelled out: After receiving the evidence, organizations need to understand it, value it, and use it to change policies or ways of thinking. There are many ways that this could go wrong, resulting in little or no impact. Leaving this out of the picture risks these crucial details being neglected.

Example 2: A conditional cash transfer program

Below is a theory of change for a program that uses cash transfers to increase child immunization rates in India.^[4] The program has two “input” actions: providing resources to keep immunization camps reliably open, and providing cash incentives to people who get vaccinated. 

What’s particularly attractive about this theory of change is how measurable each step is. For example, we can send someone to check and see if a camp is reliably open. We can create a verification system to check if incentives are delivered. We can count how many children come to the camps, and how often they return for their booster shots. If something is going wrong, we can check the assumptions that are explicitly listed (e.g. surveying parents to see if they trust the camps). 

As a result of the clear, measurable theory of change, J-PAL (The Abdul Latif Jameel Poverty Action Lab) was able to evaluate the performance of the program to come to a view on its cost-effectiveness.^[5]

The ‘how’ of theory of change

Creating a theory of change involves linking up a number of cause-effect relationships (based on scientific evidence, expert advice, or well-reasoned assumptions), to form a path from inputs to impact. It is the type of skill that is hard to teach in the form of a list of explicit instructions – rather, it is best learned by doing and by looking at real examples, and noticing what it is about them that is effective or ineffective. 

Tips for crafting a theory of change (Do’s)

1. Invest sufficiently in understanding the context of the problem: As anyone who works in the nonprofit sector will confirm, solving the world’s problems isn’t easy. Things might seem simple (e.g. ‘vaccinations save lives, so all we need to do is produce enough vaccines and we’ll save lots of lives!’), but the world is complicated and the devil is in the details (consider all the ways that the theory of change in Example 2 could go wrong). To craft a theory of change that works in reality, it’s crucial that you sufficiently understand reality. This means you need to understand exactly who the people you intend to serve are and what their wants, needs and views are (e.g. who has most control over whether a child is vaccinated – the child? their parents? public institutions like schools?). You also need to understand the barriers that have prevented people from resolving the problems they face, you need to understand who the stakeholders are, what their incentives are, and you need to sufficiently understand the broader political and economic context. People who have spent time on the ground understanding the context tend to design the most realistic theories of change.
2. Map the causal pathway backwards from impact to activities: Programs should be defined by starting with the problem and looking for the best solution, rather than by starting with a set of activities in mind and trying to find a plausible story for how those activities could solve the problem. Working backwards reduces your susceptibility to confirmation bias, because instead of asking ‘what evidence can I find that this activity causes the desired effect?’, you’re forced to ask ‘what activity does the evidence say is most effective at causing the desired effect?’
3. Question every causal step: Given how easy it is for a charitable program to have minimal impact, and given the stakes, we recommend approaching each theory of change with a healthy skepticism. For each causal step, ask yourself: Is it clear why A would cause B? How might this step fail? Can we justify how confident we are in this step?

Hallmarks of an excellent theory of change (Some more Do’s)

1. A focused suite of activities: A common way that charities become inefficient is through mission creep. The reasoning goes something like, ‘if we’re already providing mosquito nets to prevent malaria, why not also provide vitamin supplements, or distribute essential medicines?’ Sometimes adding more activities is efficient, squeezing more impact out of a similar fixed-cost base. But most of the time it isn’t. This is the case for a number of reasons; for example, more activities means that an organization’s focus is split between competing priorities. More activities means more coordination costs at the organization. Adding additional activities that are less cost-effective than the first ones will often hurt average cost-effectiveness more than the synergies help it. Meanwhile, there is an incentive for organizations to add more activities to their theory of change, because individuals, donors and undiscerning funders often think it is more impressive to have a larger, more holistic suite of programs. As a result, we have a heuristic that theories of change with a tight suite of activities tend to be stronger than those with a long list of activities that span a number of distinct interventions. 
2. Explicitly naming the evidence or assumptions behind each step: One practice we’d like to see in more theories of change, is an indication of the evidence, or the assumptions, behind each step. This makes it easier for outsiders to vet the theory of change: They can review the evidence and check whether they agree with the assumptions. It also makes it clear which steps are most in need of measurement to validate assumptions. 
3. Communicating the relative confidence in each step: The most transparent theories of change make it clear which steps are the most tenuous or liable to fail, so that both those reviewing the plan and those implementing it can focus on those steps. This can be done by color coding the arrows or boxes, and ideally can be accompanied by explicit probability estimates.
4. Making it clear who the actor is in each step: The clearest theories of change explain who is supposed to take the action in each intermediate step (e.g. instead of ‘children are brought to immunization camps’, it should be ‘parents bring children to immunization camps’). This makes the mechanics of the intervention clearer to outside readers, and makes it easier to think about and prepare for how the step could go wrong (‘why might the parents not bring their children?’).

Common mistakes to watch out for (Don’ts)

1. Not making fundamental impact the goal: The conditional cash transfer example is a good theory of change, but can it be improved? Immunization isn’t the end goal – saving lives and improving quality of life is. This may seem obvious, but it is very important to note it explicitly so that we remember to model how much disease is actually prevented, and how many lives are actually saved by increasing immunization. If we skip this step, we might (for instance) end up choosing a location that maximizes the number of people who can be immunized (e.g. an area that has a good network of vaccination clinics), rather than one that maximizes the number of lives we can counterfactually save (e.g. by picking a location with a high disease burden).  
2. Insufficiently detailed: Many theories of change make the mistake of leaving out important details in two ways:
   
   1.  Making large leaps between each step. For example:
            Publish research ⭢ lives are saved
            Raise awareness ⭢ less suffering
      By breaking these steps down into the main sub-steps, each of which depend on assumptions that we can’t be certain about, it becomes clear that these theories of change are a lot less likely to work than they initially appear. 
   2. Combining multiple major outcomes in one step. The best theories of change break out significant, independent outcomes like ‘government introduces regulation’ and ‘government enforces regulation’ into separate steps, so that the important risk that something goes wrong in between is made salient. By contrast, ‘government introduces and enforces regulation’ makes it easy to forget that it’s possible for regulation to be introduced but not enforced. 
3. Set and forget: Many organizations design a theory of change very early on, then put it in a drawer and never look at it again. Theories of change should be treated as living documents that are constantly revised as the charity tests things and gains new information. 
4. Not building your theory of change into a measurement plan: One of the key purposes of articulating a theory of change is to determine what to measure so that you can validate whether and how much impact an intervention is having. A good theory of change makes it clear what steps are the most uncertain, and so most in need of measurement. Despite this, it’s common for charities to have no measurement plan at all, and when they do, for it to not be designed with reference to the theory of change. We will discuss this further in Chapter 5.2 on monitoring and evaluation.

Theory of change for foundations

There are two places where theories of change play a major role in grantmaking foundations’ decision-making:

1. Assessing grantees
2. Defining a theory of change for the foundation itself

Assessing grantees

The most obvious place for theories of change to play a role, is that grantmakers should expect that all potential grantees have a theory of change for their organization or project, or at least that they can produce one when asked. Funders in the nonprofit space need to hold implementing actors to account on implementing an explicit, realistic and well-evidenced plan for turning funding into impact. 

Better yet, grantmakers should expect that recipient organizations use their theory of change to identify priorities for their monitoring and evaluation plan, and that they build a cost-effectiveness analysis based on the theory of change’s structure (quantifying the costs of their inputs, and the size and probability of the desired outputs, outcomes and impact).

Theory of change for the foundation

It may be a little less intuitive, but theories of change also have a place in a foundation’s decision-making by articulating the strategy for how the foundation itself will have impact. 

Direct impact of grantmaking

At first thought, it might seem like there would be one theory of change for how all foundations have impact: ‘we fund impactful activities’. In this sense, a foundation’s theory of change is the sum of each of its grantees’, with an additional input at the beginning:

In some cases, foundations can define their theory of change in terms of a specific problem (e.g. inhumane chicken farming) or a specific type of solution (e.g. plant-based alternatives) that they have decided to focus on, where the theory of change reflects their view of the best strategy for how to improve the world within this focus area:

Armed with this kind of strategically-framed theory of change, foundations can seek grantees who can make a contribution and can publicly communicate how other organizations can play a role in realizing their vision. The downside of this type of theory of change for a foundation is that it forces you to have a clear and narrow view of the problem you will focus on, and how it can best be solved. As a result, this approach isn’t suitable for all foundations, and when foundations do take this approach, they should be mindful not to become anchored to one narrow view of how to improve the world. 

Indirect foundation impact

In addition to the direct impact of providing funding, a foundation can also have broader impact through thought leadership. For example, you might publish research (like Open Philanthropy’s reports on different cause areas), write blog posts (like the Mulago Foundation), develop impact metrics (like CE’s Welfare Points) or create public cost-effectiveness analyses of charities and interventions (like GiveWell). 

Foundations can also have impact by influencing their network (which may include other high net-worth individuals, or even other foundations) to give more, or give more effectively. There are a number of other ways foundations can go about creating leverage, effectively multiplying their spending, such as by accessing government funding. 

Another way of having an impact is by setting good norms in the nonprofit sector; making it a public policy to only fund organizations who follow certain best practices (e.g. pre-committing to shut down if the M&E doesn’t show a predetermined threshold of cost-effectiveness). 

Meanwhile, it is overly narrow to only think about the impact of grant dollars in terms of the inputs that those dollars fund. For example, sometimes when a foundation gives a grant to an early stage organization, it makes the difference between them going insolvent, and them surviving long enough to scale into a massively impactful charity (like the Against Malaria Foundation). By ensuring the longevity of a highly impactful charity, the impact of the grant is not just the immediate activities covered by that money, but some percentage of all of the good that the organization goes on to do in the long term. Similarly, foundations can be active in their grantmaking, influencing how many and what type of projects come into existence. They then are partially responsible for those projects’ lifetime impact, beyond the activities they fund directly.   

In some cases, these other forms of impact end up being much larger than the direct impact of the grants themselves! But the chances of this being the case accidentally are low – it takes thought and effort. For this reason, we encourage foundations to define their own theory of change, so that they make a conscious decision about which of these other forms of impact they will work to cultivate. 

Charity Entreneurship’s theory of change for its Incubation Program

We’ve spent this post ‘talking the talk’ when it comes to theories of change, but now it’s time to ‘walk the walk’. Here is a theory of change for CE’s Incubation Program, as it stands as of July 2023:

Links to footnotes within the table above: (1), (2), (3), (4), (5), (6), (7), (8)

Note that the TOC does not exhaustively cover all necessary inputs for the incubation program to exist and have impact. For example, all of the crucial work of our operations team is absent, as is the work of the co-founder team to set the strategic direction and ensure proper functioning of each department. This is intentional, as it’s important to keep the TOC as tight as possible to increase clarity and comprehensibility. Inputs that truly go without saying need not be included – however in our experience people tend to leave out more than they should.

The TOC reveals some of the risks to our Incubation Program’s impact, for example:

• We may be at risk of exhausting the talent pool, particularly within the Effective Altruism movement, and so we need to continue to innovate in our approach to outreach. 
• Program facilitation is likely not optimal, hurting the number and quality of charities founded. Whilst we believe we are continuing to make progress on this front, it’s difficult to be certain because trends in program outcomes are hard to measure precisely (due to the small sample sizes, both in terms of the number of cohorts and number of participants per cohort). It’s also hard to know to what extent trends in program outcomes can be attributed to the facilitation itself, because other highly variable factors – like how participants’ co-founder and idea preferences end up shaking out – significantly shape how much impact each cohort produces.
• The ability of the funding landscape to support charities through the ‘valley of death’ is a bottleneck to scaling the incubation program to found more charities per year. 

Whilst the evidence we have for the impact of the Incubation Program is fairly strong, one weakness that this TOC illustrates is that several pieces of evidence are predicated on the assumption that funders always make sound decisions, when in reality, getting funding and achieving impact are not necessarily correlated and weak orgs sometimes limp along and take up resources far longer than they should. The exercise of formally documenting this TOC has reinforced for us the importance of the external impact evaluations of our more mature incubated charities that we have planned for 2024. We hope that these evaluations will significantly increase the rigor and objectivity of our M&E.

Conclusion and next steps

I hope this discussion has convinced you of the importance of nailing the theory of change for any non-profit program to be impactful. 

If you are personally involved in a non-profit program (either as a funder or an implementor): 

• Does the program have a high quality theory of change? 
• Is it publicly available?
• Is the entire organization familiar with it?
• Have you gone through the process of tearing it to shreds and then rebuilding it better than before?
• Is your organization willing to stop supporting the program if the theory of change looks too weak?

If the answer to any of these questions is ‘no’, you have work to do. 

If you are interested in applying to participate in the Incubation Program: Applications are now open. You can learn more about the process, or begin the application process here.

If you’d like to express interest in participating in the Foundation Program: Reach out to Will Troy.

If you’d like to be notified when our book on effective grantmaking is published, you can let us know here.

Special thanks to Leonie Falk for your help in pressure-testing our theory of change.
 

1. ^{^}

   Practitioners disagree on the exact distinction between ‘outputs’ and ‘outcomes’. In our experience, the distinction is unimportant. We recommend focusing on the ‘cause-and-effect mapping’ concepts and not the terminology used to describe it.

2. ^{^}

   A clear theory of change can serve as the blueprint for a program’s first cost-effectiveness analysis by laying out the activities that make up the costs, and the various causal steps towards impact, each of which have an effect size and/or probability of success based on the evidence and assumptions for those steps.

3. ^{^}

    Happier Lives Institute, “HLI Has Hatched: Strategy Update after the Charity Entrepreneurship Incubation Program,” Oct. 29, 2019, accessed Feb. 20, 2023, https://www.happierlivesinstitute.org/2019/10/29/hli-has-hatched-strategy-update-after-the-charity-entrepreneurship-incubation-program/. 

4. ^{^}

   J-PAL, “Theory of Change,” Course lecture, J-PAL/CLEAR South Asia at IFMR, Delhi, July 2017, accessed Feb. 20, 2023, https://www.povertyactionlab.org/sites/default/files/Lecture%202-THEORY%20OF%20CHANGE.pdf

5. ^{^}

   Abhijit Banerjee et. al., “Improving Immunization Rates Through Regular Camps and Incentives in India,” J-PAL, 2007, accessed Feb. 20, 2023, https://www.povertyactionlab.org/evaluation/improving-immunization-rates-through-regular-camps-and-incentives-india