Requesting feedback: proposal for a new nonprofit with potential to become highly effective

By Marshall @ 2022-09-13T11:28 (+55)

TL;DR

I make the case for funding a new initiative that addresses an important, tractable, and neglected problem with potential to reach GiveWell-level cost-effectiveness. Please review this and share your thoughts!

• Millions of people die every year from preventable conditions, especially in low- and middle-income countries (LMICs).
• Some of these deaths can be avoided by improving health workers’ skills and knowledge.
• Working with a nonprofit, I’ve been piloting a way to train health workers online with a scrappy WordPress-based MVP, and still getting good results. 
• I’m looking for funding to create and scale a solution that’s 10X better than anything that exists.

Why I’m sharing this

I’m pretty new to EA and I’d like to submit a version of this to EA organizations for funding. I’m seeking feedback to help make this as clear, logical, and succinct as possible.

Importance: what is the scale of problems that can be addressed with health worker training?

Every year, there are tens of millions of deaths that can be averted with simple interventions, including:

• 2.9 million deaths of newborn babies readily preventable with standard care,
• 1.5 million deaths caused by diarrheal diseases; many of which can be averted with improved sanitation practices and low-cost treatment, 
• 2.5 million deaths caused by lower respiratory infections; some of which can be prevented with appropriate diagnosis and treatment, and
• 10.8 million deaths caused by high blood pressure, which is inexpensively controlled with generic medications.

Pandemics are an equally important global problem. COVID-19 has caused an estimated 17.5 million deaths. Unfortunately, experts agree that COVID-19 isn’t a worst-case scenario: future pandemics could pose an existential threat to humanity.

These problems and their causes have an uneven global distribution. Newborn deaths, lower respiratory infections, and diarrheal disease, in particular, are concentrated in LMICs. Moreover, for a variety of reasons, LMICs are particularly susceptible to epidemics. Solutions to these problems must be implemented in LMICs.

Tractability: how can online health worker training solve these problems?

There is ample evidence that health worker (HW) training can have benefits on its own or when combined with other interventions. 

• My analysis of existing research studies shows that training HWs to properly care for newborn babies is likely to be highly cost-effective, with an average cost of $59 per DALY averted ($100 per DALY averted is sometimes cited as a benchmark for highly effective interventions). Please see the full forum post for the details of this analysis.
• Management of childhood illness training, which covers simple interventions (such as appropriately diagnosing and treating lower respiratory infections and diarrhea), measurably improves HW clinical skills.
• A recent study estimated that properly training HWs and giving them personal protective equipment would have saved $7.2 billion in just the first six months of the COVID-19 pandemic.
• The decisions that HWs make early in a disease outbreak can save countless lives. Notably, a doctor who acted quickly to quarantine a sick traveler is credited with stopping the spread of Ebola into Nigeria in 2014. HWs need to be well trained to be ready to make these critical decisions at any time.

There is also emerging evidence that online learning is an effective way to train HWs.

• Online neonatal care training leads to significant knowledge and skills gains among HWs.
• The infection control training I’ve been working on has high completion rates, learning gains comparable to those seen in more resource-intensive in-person training, and very positive learner feedback.
• Recent research shows that a short online training in blood pressure measurement improves HWs’ clinical skills.
• Modeling indicates that online HW training could be highly cost-effective. Even if online training is much less effective than in-person training on a per-HW basis, its potential for massive scale could still make it a “best buy.” 

Finally, training can be used to address many different problems - it’s a cross-cutting solution. For example, a single platform could provide high-quality training on management of childhood illness and newborn care, while also rapidly updating HWs with the latest guidance on emerging pathogens with pandemic potential.

Neglectedness: what gaps could this solution fill?

Most aspects of health workforce development require greater attention and investment. 

• As of 2013, there was a global shortage of 17.4 million HWs, with a shortfall of 4.2 million HWs in Africa alone. 
• A lack of essential skills and knowledge is a persistent but understudied problem in the health workforce. The available evidence paints a grim picture. For example:
  • in a referral hospital in Nigeria, over 70% of HWs have poor knowledge of blood pressure measurement,
  • HWs caring for sick children in four African countries had very low rates of adherence to evidence-based protocols for diagnosing the most life-threatening conditions (though adherence was higher among those who were recently trained), and
  • HWs who take our infection control course enter with low baseline knowledge in the subject.
• Gaps in skills and knowledge are associated with insufficient training. For example, a recent study found that only ~25% of low-income countries surveyed had ongoing infection control training for HWs who are in the workforce.
• Global investment in HW training is fragmented, which prevents the most promising interventions (such as digital HW training) from moving past pilot stages and realizing economies of scale.
• A lack of high-quality research on HW training, particularly in LMICs, prevents us from even developing a complete understanding of the problem.

More generally, I’ve searched extensively for high-quality online training targeted to HWs in LMICs. There are some great resources, but there is no “Duolingo for health care,” and I believe the world is ready for that. The time is right, given ongoing massive growth in mobile device ownership and internet access in many of the countries with the highest burden of preventable disease.

Progress to date

Since April 2021, working with the nonprofit Resolve to Save Lives, I have been creating and delivering mobile-first, simple, engaging, and effective online HW training. We currently use a WordPress-based MVP to deliver the training.

• The learning process is inspired by Duolingo and based on best practices from education research. HWs learn by responding to questions and receiving feedback in short modules that present realistic clinical cases.
• We work closely with governments to offer the training in existing health facilities, so we don’t incur high costs of setting up new hospitals and health centers. 
• We’ve already had thousands of HWs from across Africa enroll in courses.
• Results of an initial pilot of an infection control course were published in BMC Public Health.
• Completion rates of a follow-up pilot reached 65%, more than 10 times higher than the average completion rates of free, optional, self-paced courses taken in LMICs.
• There were substantial and significant knowledge gains among HWs who completed the infection control course.

Funding request and next steps

[Note: this section has been updated based on feedback; we make a small-scale funding request for a pilot, rather than a larger request for full project implementation.]

We seek funding of $250,000 for 12 months. We’ll use this catalytic funding to run a small-scale pilot, reaching >1,000 HWs in 1-2 countries in Africa. In this pilot, we will assess the feasibility of the solution and measure short-term outcomes (learning gains, completion, and learner feedback). Our goal is to use a lean startup approach to learn as much as possible and accelerate toward a larger-scale, rigorous (and more costly) evaluation of the health impacts of the intervention.

We will start with courses in English that align with international (World Health Organization) guidance, and launch in anglophone Africa before expanding to additional regions and languages. For the foreseeable future, we will focus on HWs who work in primary care - including doctors, nurses, midwives, and community health workers - who form the backbone of the health system and most frequently diagnose and treat the preventable diseases discussed in this piece.

Contact me for more detailed information such as a budget and key milestones. 

Addressing skepticism and frequently asked questions

This project is clearly not risk-free. HW training is not as well-studied as other interventions such as the GiveWell top charities. Here are some questions I’ve fielded about this project, along with my answers.

There are a lot of other things that contribute to these preventable deaths, such as a lack of access to medications and equipment. Aren’t those more important problems? How can we be sure that HW training is the right solution to invest in?

It’s absolutely true that this problem has many root causes. I’m not making the broad claim that training will put an end to readily-preventable deaths. Instead, I’m making the more narrow claim that it is a cost-effective way to avert some of these deaths. Most health systems strengthening activities are quite expensive and don’t scale particularly well; online HW training merits more attention because it looks like an exception to this rule.

Isn’t it presumptuous to state that a lack of HW knowledge or skills leads to deaths? Don’t we need to prove that before we invest a lot in training interventions?

As discussed above, the best available evidence on this question comes from training HWs in newborn care, where several different studies have shown that interventions cost-effectively save lives. These studies support the claim that inadequate training contributes to unnecessary deaths. If this project is funded, we would be able to begin to develop a better understanding of whether this is true of topic areas other than newborn care.

Shouldn’t you just focus on training HWs in newborn care, since that’s the topic with the best cost-effectiveness data? 

This sounds perfectly reasonable but I don’t think it accounts for how educational technology products achieve widespread adoption in the real world. To me, this is a bit like suggesting that Netflix remove all of its other programming to focus on Stranger Things. The most widely used online learning platforms all offer a large amount of content (or let users contribute their own content). While newborn care training is the first course on our roadmap, I think it would be very difficult to maximize impact without a critical mass of content. Creating additional courses is not nearly as expensive as building a great learning platform and acquiring learners. If we can create a trusted, engaging learning resource with valuable content covering many different conditions, it will be much more valuable to the HWs themselves than a solution focused on just one cause of death. 

Why not just choose existing training tools and interventions and point HWs to those?

Simply put, because most existing solutions are not designed for HWs in LMICs, and there isn’t much evidence that they work for training this audience. For example, the World Health Organization puts out guidance on topics ranging from newborn care to infection control for COVID-19. This guidance is packaged in lengthy and dense PDFs. It’s not realistic to expect frontline HWs to wade through all of this guidance to find the few paragraphs or pages in each document relevant to their work. Even though we’ve run pilots on a shoestring so far, the user-centered approach we are taking is already yielding better results than those existing solutions. 

I don’t think online learning works. In fact, I think it’s terrible. How can you justify more investment in it?

Some hesitation about online learning is certainly warranted because there’s a lot of bad online training out there. If you are skeptical about online learning in general, I’d recommend my previous writing that analyzes the challenges we face and solutions we can leverage to create effective online HW training. To summarize, solutions have to be carefully designed for efficacy and tailored to the learning needs of the target audience. It’s not surprising that badly designed online learning doesn’t deliver!

Shouldn’t we invest in the pipeline of new HWs rather than training existing HWs?

Training new HWs is expensive, resource-intensive, and heavily regulated. I know this from my first-hand experience in helping to launch a medical school in Rwanda. Continuing education has a better landscape for innovation: the barriers to entry are lower, accreditation is simpler, and the feedback cycles are shorter. Although there’s a great need in both areas, there’s no reason that this innovation can’t get started in the continuing education space and subsequently contribute to improving and simplifying the process of training new HWs.

How does your argument account for uncertainty?

For a more in-depth discussion of some limitations and cost-effectiveness modeling that accounts for different levels of efficacy of online training, I would refer readers to an earlier post I wrote on this topic. 

Expressing gratitude

I’d like to thank Alex Chalk and an anonymous member of the EA community for providing great feedback on early drafts of this piece. I contacted Holly Kristensen after seeing this post and also received great feedback from her. The ideas (and any errors) in this piece are mine and do not necessarily reflect the opinions of these generous reviewers. I’m extremely grateful to the team at Resolve to Save Lives for supporting this work in a number of critical ways.