Nailing the basics – Theories of change

By Aidan Alexander, CE @ 2023-07-16T06:40 (+225)

Why write a post about theories of change?

As participants in a movement with ‘effective’ in its name, it’s easy to think of ourselves as being above falling for the most common mistakes made in the non-profit sector more broadly. We’re grappling with the cutting-edge stuff, not the basics, right?

But we need to beware of this kind of complacency. It’s crucial to nail the basics, like theories of change, and far fewer EA organizations do this than you’d expect. That includes Charity Entrepreneurship, the organization I work at, where we teach participants in our Incubation Program and Foundation Program about the importance of theories of change, and yet until today we didn’t even have our own theory of change on our website!

That’s why we decided to share this post on theories of change – potentially the first of a series on ‘nailing the basics’ of effective non-profit work, if people are interested. This post is mostly made up of an excerpt from our book on effective grantmaking (‘How to Launch a High-Impact Foundation’), followed by a discussion of the theory of change for our Incubation Program. 

TL;DR

The ‘what’ and ‘why’ of a theory of change

Building something complicated without an explicit plan is risky. From skyscrapers to software, ambitious projects need detailed blueprints. When building an effective nonprofit organization, the theory of change is that blueprint.

In the same way that you wouldn’t hire a builder who intends to build you a home without a blueprint, you shouldn’t fund a leader who intends to build an organization or launch a project without a theory of change. If the builder planned to create something particularly cutting-edge, like a home made entirely out of recycled cardboard that doesn’t need heating or cooling, you’d be particularly adamant that the builder explains exactly how this is possible and what evidence or assumptions it depends on. The same applies when it comes to funding an ambitious project that aims to improve the world. 

A theory of change explicitly articulates the logical steps of a plan to achieve a specific goal. It is generally represented in the form of a cause-and-effect diagram. A well-designed theory of change allows an organization to clearly communicate its activities and how they lead to the desired outcomes, and then to impact.[1]

In fact, theories of change are more important for charities than business models are for businesses, because businesses have better feedback loops. If a for-profit is based on a bad idea, or is badly executed, it will see poor revenue and it will soon go out of business. On the other hand, if a charity is ineffective, it may limp along for years without having any impact, squandering limited funding and talent in the process.

Foundations deciding which opportunities to fund should expect a clear theory of change as table-stakes for potential grantees. In addition to spelling out how the potential grantee intends to convert your funding into impact, theories of change will allow you to determine how confident you can be that impact will be achieved. Is the causal chain tight, or convoluted? Are the steps well substantiated by evidence, or based on tenuous assumptions? Can any assumptions be easily tested? What is the crux that the project’s success depends on? A smart nonprofit will use its theory of change to identify its priorities for measurement and evaluation, and as the blueprint for its first cost-effectiveness analysis,[2] which foundations can then use to inform funding decisions.

Real world examples

Example 1: A research organization

Imagine an organization that wants to use research to make a difference. Without an explicit theory of change, their implicit mental model of how they might make an impact may look something like this:

Review literature ⭢ Find insights that seem useful ⭢ Publish them

This is not a good theory of change. It doesn’t properly outline the goal, it doesn’t explain how the actions will lead to that goal, and it doesn’t explain how they will measure what they are doing. Presumably, the final goal is to impact living beings positively, not just publish papers. Even if the organization works very hard and puts out many widely cited papers, it’s possible that it will not make any difference. 

Compare this implicit model to an explicit theory of change published by a young charity after they went through one of Charity Entrepreneurship’s early incubation programs:


[3]

On the left, you can see the charity's core research agenda at a glance (the inputs/activities). On the right, you can see some measurable metrics such as money moved, new projects, and hires made (intermediate outcomes) which lead to the final impact – happier lives.

This is definitely an improvement on the implicit mental model: 

Having developed this theory of change, they will be more likely to give adequate focus to ensuring that they don’t just publish research, but that the right people read it and change their actions (e.g. fund more impactful organizations, start more impactful projects, or measure impact in a more appropriate manner). 

That being said, this theory of change has significant room for improvement. In particular, the intermediate steps between communicating the research and organizations changing their actions ought to be spelled out: After receiving the evidence, organizations need to understand it, value it, and use it to change policies or ways of thinking. There are many ways that this could go wrong, resulting in little or no impact. Leaving this out of the picture risks these crucial details being neglected.

Example 2: A conditional cash transfer program

Below is a theory of change for a program that uses cash transfers to increase child immunization rates in India.[4] The program has two “input” actions: providing resources to keep immunization camps reliably open, and providing cash incentives to people who get vaccinated. 

What’s particularly attractive about this theory of change is how measurable each step is. For example, we can send someone to check and see if a camp is reliably open. We can create a verification system to check if incentives are delivered. We can count how many children come to the camps, and how often they return for their booster shots. If something is going wrong, we can check the assumptions that are explicitly listed (e.g. surveying parents to see if they trust the camps). 

As a result of the clear, measurable theory of change, J-PAL (The Abdul Latif Jameel Poverty Action Lab) was able to evaluate the performance of the program to come to a view on its cost-effectiveness.[5]

The ‘how’ of theory of change

Creating a theory of change involves linking up a number of cause-effect relationships (based on scientific evidence, expert advice, or well-reasoned assumptions), to form a path from inputs to impact. It is the type of skill that is hard to teach in the form of a list of explicit instructions – rather, it is best learned by doing and by looking at real examples, and noticing what it is about them that is effective or ineffective. 

Tips for crafting a theory of change (Do’s)

  1. Invest sufficiently in understanding the context of the problem: As anyone who works in the nonprofit sector will confirm, solving the world’s problems isn’t easy. Things might seem simple (e.g. ‘vaccinations save lives, so all we need to do is produce enough vaccines and we’ll save lots of lives!’), but the world is complicated and the devil is in the details (consider all the ways that the theory of change in Example 2 could go wrong). To craft a theory of change that works in reality, it’s crucial that you sufficiently understand reality. This means you need to understand exactly who the people you intend to serve are and what their wants, needs and views are (e.g. who has most control over whether a child is vaccinated – the child? their parents? public institutions like schools?). You also need to understand the barriers that have prevented people from resolving the problems they face, you need to understand who the stakeholders are, what their incentives are, and you need to sufficiently understand the broader political and economic context. People who have spent time on the ground understanding the context tend to design the most realistic theories of change.
  2. Map the causal pathway backwards from impact to activities: Programs should be defined by starting with the problem and looking for the best solution, rather than by starting with a set of activities in mind and trying to find a plausible story for how those activities could solve the problem. Working backwards reduces your susceptibility to confirmation bias, because instead of asking ‘what evidence can I find that this activity causes the desired effect?’, you’re forced to ask ‘what activity does the evidence say is most effective at causing the desired effect?’
  3. Question every causal step: Given how easy it is for a charitable program to have minimal impact, and given the stakes, we recommend approaching each theory of change with a healthy skepticism. For each causal step, ask yourself: Is it clear why A would cause B? How might this step fail? Can we justify how confident we are in this step?

Hallmarks of an excellent theory of change (Some more Do’s)

  1. A focused suite of activities: A common way that charities become inefficient is through mission creep. The reasoning goes something like, ‘if we’re already providing mosquito nets to prevent malaria, why not also provide vitamin supplements, or distribute essential medicines?’ Sometimes adding more activities is efficient, squeezing more impact out of a similar fixed-cost base. But most of the time it isn’t. This is the case for a number of reasons; for example, more activities means that an organization’s focus is split between competing priorities. More activities means more coordination costs at the organization. Adding additional activities that are less cost-effective than the first ones will often hurt average cost-effectiveness more than the synergies help it. Meanwhile, there is an incentive for organizations to add more activities to their theory of change, because individuals, donors and undiscerning funders often think it is more impressive to have a larger, more holistic suite of programs. As a result, we have a heuristic that theories of change with a tight suite of activities tend to be stronger than those with a long list of activities that span a number of distinct interventions. 
  2. Explicitly naming the evidence or assumptions behind each step: One practice we’d like to see in more theories of change, is an indication of the evidence, or the assumptions, behind each step. This makes it easier for outsiders to vet the theory of change: They can review the evidence and check whether they agree with the assumptions. It also makes it clear which steps are most in need of measurement to validate assumptions. 
  3. Communicating the relative confidence in each step: The most transparent theories of change make it clear which steps are the most tenuous or liable to fail, so that both those reviewing the plan and those implementing it can focus on those steps. This can be done by color coding the arrows or boxes, and ideally can be accompanied by explicit probability estimates.
  4. Making it clear who the actor is in each step: The clearest theories of change explain who is supposed to take the action in each intermediate step (e.g. instead of ‘children are brought to immunization camps’, it should be ‘parents bring children to immunization camps’). This makes the mechanics of the intervention clearer to outside readers, and makes it easier to think about and prepare for how the step could go wrong (‘why might the parents not bring their children?’).

Common mistakes to watch out for (Don’ts)

  1. Not making fundamental impact the goal: The conditional cash transfer example is a good theory of change, but can it be improved? Immunization isn’t the end goal – saving lives and improving quality of life is. This may seem obvious, but it is very important to note it explicitly so that we remember to model how much disease is actually prevented, and how many lives are actually saved by increasing immunization. If we skip this step, we might (for instance) end up choosing a location that maximizes the number of people who can be immunized (e.g. an area that has a good network of vaccination clinics), rather than one that maximizes the number of lives we can counterfactually save (e.g. by picking a location with a high disease burden).  
  2. Insufficiently detailed: Many theories of change make the mistake of leaving out important details in two ways:
    1.  Making large leaps between each step. For example:
            Publish research ⭢ lives are saved
            Raise awareness ⭢ less suffering
      By breaking these steps down into the main sub-steps, each of which depend on assumptions that we can’t be certain about, it becomes clear that these theories of change are a lot less likely to work than they initially appear. 
    2. Combining multiple major outcomes in one step. The best theories of change break out significant, independent outcomes like ‘government introduces regulation’ and ‘government enforces regulation’ into separate steps, so that the important risk that something goes wrong in between is made salient. By contrast, ‘government introduces and enforces regulation’ makes it easy to forget that it’s possible for regulation to be introduced but not enforced. 
  3. Set and forget: Many organizations design a theory of change very early on, then put it in a drawer and never look at it again. Theories of change should be treated as living documents that are constantly revised as the charity tests things and gains new information. 
  4. Not building your theory of change into a measurement plan: One of the key purposes of articulating a theory of change is to determine what to measure so that you can validate whether and how much impact an intervention is having. A good theory of change makes it clear what steps are the most uncertain, and so most in need of measurement. Despite this, it’s common for charities to have no measurement plan at all, and when they do, for it to not be designed with reference to the theory of change. We will discuss this further in Chapter 5.2 on monitoring and evaluation.

Theory of change for foundations

There are two places where theories of change play a major role in grantmaking foundations’ decision-making:

  1. Assessing grantees
  2. Defining a theory of change for the foundation itself

Assessing grantees

The most obvious place for theories of change to play a role, is that grantmakers should expect that all potential grantees have a theory of change for their organization or project, or at least that they can produce one when asked. Funders in the nonprofit space need to hold implementing actors to account on implementing an explicit, realistic and well-evidenced plan for turning funding into impact. 

Better yet, grantmakers should expect that recipient organizations use their theory of change to identify priorities for their monitoring and evaluation plan, and that they build a cost-effectiveness analysis based on the theory of change’s structure (quantifying the costs of their inputs, and the size and probability of the desired outputs, outcomes and impact).

Theory of change for the foundation

It may be a little less intuitive, but theories of change also have a place in a foundation’s decision-making by articulating the strategy for how the foundation itself will have impact. 

Direct impact of grantmaking

At first thought, it might seem like there would be one theory of change for how all foundations have impact: ‘we fund impactful activities’. In this sense, a foundation’s theory of change is the sum of each of its grantees’, with an additional input at the beginning:

In some cases, foundations can define their theory of change in terms of a specific problem (e.g. inhumane chicken farming) or a specific type of solution (e.g. plant-based alternatives) that they have decided to focus on, where the theory of change reflects their view of the best strategy for how to improve the world within this focus area:

Armed with this kind of strategically-framed theory of change, foundations can seek grantees who can make a contribution and can publicly communicate how other organizations can play a role in realizing their vision. The downside of this type of theory of change for a foundation is that it forces you to have a clear and narrow view of the problem you will focus on, and how it can best be solved. As a result, this approach isn’t suitable for all foundations, and when foundations do take this approach, they should be mindful not to become anchored to one narrow view of how to improve the world. 

Indirect foundation impact

In addition to the direct impact of providing funding, a foundation can also have broader impact through thought leadership. For example, you might publish research (like Open Philanthropy’s reports on different cause areas), write blog posts (like the Mulago Foundation), develop impact metrics (like CE’s Welfare Points) or create public cost-effectiveness analyses of charities and interventions (like GiveWell). 

Foundations can also have impact by influencing their network (which may include other high net-worth individuals, or even other foundations) to give more, or give more effectively. There are a number of other ways foundations can go about creating leverage, effectively multiplying their spending, such as by accessing government funding. 

Another way of having an impact is by setting good norms in the nonprofit sector; making it a public policy to only fund organizations who follow certain best practices (e.g. pre-committing to shut down if the M&E doesn’t show a predetermined threshold of cost-effectiveness). 

Meanwhile, it is overly narrow to only think about the impact of grant dollars in terms of the inputs that those dollars fund. For example, sometimes when a foundation gives a grant to an early stage organization, it makes the difference between them going insolvent, and them surviving long enough to scale into a massively impactful charity (like the Against Malaria Foundation). By ensuring the longevity of a highly impactful charity, the impact of the grant is not just the immediate activities covered by that money, but some percentage of all of the good that the organization goes on to do in the long term. Similarly, foundations can be active in their grantmaking, influencing how many and what type of projects come into existence. They then are partially responsible for those projects’ lifetime impact, beyond the activities they fund directly.   

In some cases, these other forms of impact end up being much larger than the direct impact of the grants themselves! But the chances of this being the case accidentally are low – it takes thought and effort. For this reason, we encourage foundations to define their own theory of change, so that they make a conscious decision about which of these other forms of impact they will work to cultivate. 

Charity Entreneurship’s theory of change for its Incubation Program

We’ve spent this post ‘talking the talk’ when it comes to theories of change, but now it’s time to ‘walk the walk’. Here is a theory of change for CE’s Incubation Program, as it stands as of July 2023:

Links to footnotes within the table above: (1)(2)(3)(4)(5)(6)(7)(8)

Note that the TOC does not exhaustively cover all necessary inputs for the incubation program to exist and have impact. For example, all of the crucial work of our operations team is absent, as is the work of the co-founder team to set the strategic direction and ensure proper functioning of each department. This is intentional, as it’s important to keep the TOC as tight as possible to increase clarity and comprehensibility. Inputs that truly go without saying need not be included – however in our experience people tend to leave out more than they should.

The TOC reveals some of the risks to our Incubation Program’s impact, for example:

Whilst the evidence we have for the impact of the Incubation Program is fairly strong, one weakness that this TOC illustrates is that several pieces of evidence are predicated on the assumption that funders always make sound decisions, when in reality, getting funding and achieving impact are not necessarily correlated and weak orgs sometimes limp along and take up resources far longer than they should. The exercise of formally documenting this TOC has reinforced for us the importance of the external impact evaluations of our more mature incubated charities that we have planned for 2024. We hope that these evaluations will significantly increase the rigor and objectivity of our M&E.

Conclusion and next steps

I hope this discussion has convinced you of the importance of nailing the theory of change for any non-profit program to be impactful. 

If you are personally involved in a non-profit program (either as a funder or an implementor): 

If the answer to any of these questions is ‘no’, you have work to do. 

If you are interested in applying to participate in the Incubation Program: Applications are now open. You can learn more about the process, or begin the application process here.

If you’d like to express interest in participating in the Foundation Program: Reach out to Will Troy.

If you’d like to be notified when our book on effective grantmaking is published, you can let us know here.

Special thanks to Leonie Falk for your help in pressure-testing our theory of change.
 

  1. ^

    Practitioners disagree on the exact distinction between ‘outputs’ and ‘outcomes’. In our experience, the distinction is unimportant. We recommend focusing on the ‘cause-and-effect mapping’ concepts and not the terminology used to describe it.

  2. ^

    A clear theory of change can serve as the blueprint for a program’s first cost-effectiveness analysis by laying out the activities that make up the costs, and the various causal steps towards impact, each of which have an effect size and/or probability of success based on the evidence and assumptions for those steps.

  3. ^

     Happier Lives Institute, “HLI Has Hatched: Strategy Update after the Charity Entrepreneurship Incubation Program,” Oct. 29, 2019, accessed Feb. 20, 2023, https://www.happierlivesinstitute.org/2019/10/29/hli-has-hatched-strategy-update-after-the-charity-entrepreneurship-incubation-program/. 

  4. ^

    J-PAL, “Theory of Change,” Course lecture, J-PAL/CLEAR South Asia at IFMR, Delhi, July 2017, accessed Feb. 20, 2023, https://www.povertyactionlab.org/sites/default/files/Lecture%202-THEORY%20OF%20CHANGE.pdf

  5. ^

    Abhijit Banerjee et. al., “Improving Immunization Rates Through Regular Camps and Incentives in India,” J-PAL, 2007, accessed Feb. 20, 2023, https://www.povertyactionlab.org/evaluation/improving-immunization-rates-through-regular-camps-and-incentives-india


Jona @ 2023-07-16T16:36 (+20)

Thanks for creating this post! Sharing some thoughts on the topic based on my experience creating and redteaming theories of change (ToCs) with various EA orgs (partly echoing your observations and partly adding new points; Two concrete project examples can be found here and here).

  1. Neglectedness of ToC work (basically echoing your claim). Due to the non-pressing nature and required senior input, ToC/strategy work seems to get deprioritized very often, e.g., I have been deprioritizing updating our ToC for three months due to more pressing work. I think the optimal time spent thinking about your ToC/prioritization/strategy depends on the maturity of your project and is hard to get right, but based on my experience, most of us spend too little time on it (just like we tend to spend too little time exploring our career options as it is worth to invest 800 hours in our career planning when we can increase its impact by 1% in expectation). Assuming your org has ten staff, which work 200 days a year for 8 hours per day, you would want to invest 16k hours in figuring out how staff spends their time best if it is likely to result in a 1% impact increase
  2. More than one ToC. I think most orgs should have a ToC on the org, team and individual level as well as for each main program/activity. It seems not optimal to work on something without even having thought through how this will change the world for at least 3min (and if there are alternatives, how you could achieve the same with less work)
  3. Different levels of granularity. Depending on the purpose and the context of your ToC, you can have a three-row ToC (e.g., for small projects you are exploring), a flow-chart (e.g., to communicate the ToC of your org clearly, see examples in this post) and/or an exhaustive document showing lots of reasoning transparency, alternatives you considered etc. (e.g., to lay out the ToC of your research agenda)
  4. Developing a ToC. One simplified approach to develop a ToC on an org level which worked well with some clients but always needs tailoring looks very roughly like this: (1) Map out all potential sources of value of today (and potentially in the future), (2) Prioritize them, (3) Create a flow chart for the most promising source of value (potentially include other sources of value or create several flow charts), (4) Think through the flow of impact/value end to end as a sanity check, (5) collect data (e.g., talk to experts, run small experiments) to reduce uncertainties, (6) re-iterate. See more here
  5. Strategic implications and influencing decisions (previously mentioned but I think this is a point many people found useful and is not stressed enough in typical ToC literature). Your ToC should inform key decisions and ultimately how you allocate resources (your staff's time or money). I never experienced that we were certain about all the sources of value and the causal relationships between each step or did not have a hypothesis on how to have even more impact with an adapted or new program. So the ToC work always had at least some implications for allocating time. One great example is the Fish Welfare Initiative, which included resolving uncertainties around their ToC in their yearly priorities (See slides 13 and 24)
  6. Areas for improvement. (1) Trying to map every casual pathway and not focusing on the most important ones, (2) Deferring too much to others on what they perceive as valuable, e.g., the target group, and not doing enough first principles/independent thinking and data collection, (3) Not considering counterfactuals at all and/or not considering that there are likely several counterfactual worlds, and (4) Not laying out key assumptions and uncertainties (previously mentioned in the post but seems valuable to highlight that this also reflects my experience) among other things

Note that I likely have a significant sample bias, as organizations are unlikely to reach out to me if they have enough time to think through their ToC. Additionally, please read this as "random thoughts which came to Jona's mind when reading the article" and not as "these are the X main things EA orgs get wrong based on a careful analysis". I expect to update my views as I learn more

Aidan Alexander @ 2023-07-16T17:51 (+4)

Good stuff Jona! I agree on all fronts. 

Re: #2, at Charity Entrepreneurship for example, we should have ToCs for our Incubation Program, Grantmaking Program and Research Training Program, but we don't yet. We have a fairly polished one for the Incubation Program, and a few different ones drafted for the new Research Program we're planning, but we haven't written one down for our Grantmaking Program, so here I am again not practicing what I preach. Looks like we have work to do :) 

RasaPetrauskaite @ 2023-07-19T19:59 (+1)

Theory of change discussions are so important! In our non-profit, we frequently compare how our theory of change is in relation to the US Civil Rights Movement, global female suffrage movement, equal marriage movement, and other movements that were successful in achieving their goals. I think it’s important to make sure we seem to be on track to achieving our goals. One of our goals is abolishing animal agriculture world-wide. I think other orgs in the space of animal welfare/ animal rights can benefit from making sure they are on track to achieve in an effective and efficient way their goals.