Vox on cash transfers vs "graduation programs" for fighting extreme poverty

By Lizka @ 2022-07-07T15:09 (+29)

This is a linkpost to https://www.vox.com/future-perfect/23152657/poverty-cash-graduation-ultra-poor-brac

[Edit: I think readers should look at Stephen's comment below, which points out key weaknesses of the article and reasons to doubt its conclusions.]

Sigal Samuel from Vox's Future Perfect recently published an article I really appreciated, both because I learned some new things and because it seriously discusses important questions and overcomes the lack of newsworthiness^[1] of long-running studies: "What’s the best way to help extremely poor people? After 20 years, the evidence is in." (Subtitle: Is it really useful to “teach a person to fish” or should you just give them the damn fish already?) 

The article discusses evidence for the effectiveness of cash transfer programs and how that compares to "ultra-poor graduation programs," where 

the idea [is] that offering a combo of assets and training and cash — instead of just, say, cash — can trigger a virtuous cycle that ultimately helps ultra-poor people escape poverty. For example, you can offer people livestock plus training on how to make money off that livestock plus a bit of cash to sustain them while they get things up and running.

Some takeaways

I'm jotting down some takeaways, with the caveats that I threw this link-post together really fast, so this could easily contain errors or misunderstandings (I'd appreciate corrections in the comments!), and that I haven't checked the cited studies myself — I'm just summarizing some parts of the Vox article.

• The evidence on graduation programs is very good, even after 10 years (other programs will sometimes seem very promising for the first few years, but less promising after a while, in part because control groups catch up)
• But it's possible that the potential of the programs is over-stated because the existing trials were with some of the people who were easiest to help in such a way, so this approach is harder to generalize. Similar programs in other places may fail because the local markets are too unstable, there aren't easily identifiable appropriate groups of people to work with on this, or the programs themselves are too expensive (e.g. if it's more expensive to pay local residents to run the program).
• In general, there isn't enough understanding of when one program is better than another, and these things do vary a lot.
• It's also possible that cash transfer programs, especially one's like GiveDirectly's, "geographically saturate"^[2] an area in a way that has spillover effects which, notably, could make control groups catch up to cash transfer recipients over time (making the program seem worse than it actually is).
• "When you talk to people in the pro-graduation camp and people in the pro-cash camp, you start to realize a funny thing: These two camps are actually moving closer and closer together over time. The gap between 'teach a man to fish' and 'give a man a fish' is narrowing." There are in-between options that might (sometimes) be better than both simple cash transfer programs and full graduation programs; cash transfers along with education might be a promising version of a "minimum viable" graduation program — and something that's easier to scale.

I'd be interested in other people's takes on the article, and I've pasted some excerpts below. 

Excerpts

The intro: 

If you want to fight poverty, you probably intuitively feel that the worst-off people are the ones who should be prioritized. As difficult as it is to live on a few bucks a day, someone who’s living on just $1.90 a day clearly has it worse, and it makes sense to think you should try extra hard to help the poorest of the poor.

It’s a big moral problem, then, that a lot of anti-poverty programs fail to successfully do that.

That problem has bothered BRAC, a major international development charity based in Bangladesh, since the 1990s. Back then, the charity was working on voguish anti-poverty programs. Microfinance was all the rage, but it was becoming clear that microloans weren’t reaching the poorest households. Nobody wanted to lend to them because who knew if they could pay back the loan? And the poorest households often didn’t want to borrow because they weren’t confident that they could figure out how to turn a profit and repay.

[...]

The BRAC team decided they needed to try something new if they wanted to lastingly improve life for the worst-off — the “ultra-poor,” as they put it. So in the early 2000s, they went into village after village in Bangladesh, deliberately looked for the poorest people, and talked to them. And what they realized was that the ultra-poor aren’t only poor in terms of cash — they also lack knowledge about how to invest cash, lack confidence in themselves, and lack social ties to the broader community.

“We started realizing that it’s not going to be a simple sort of solution,” Abed said. “It’s going to have to be a package of things, because it has to address multiple vulnerabilities. So then there was this idea of a ‘big push’ investment.”

That “big push” is the idea that offering a combo of assets and training and cash — instead of just, say, cash — can trigger a virtuous cycle that ultimately helps ultra-poor people escape poverty. For example, you can offer people livestock plus training on how to make money off that livestock plus a bit of cash to sustain them while they get things up and running. This premise became the bedrock of what BRAC called the “ultra-poor graduation program,” which aims to “graduate” recipients out of extreme poverty.

On studies over time: 

Sometimes an anti-poverty experiment will show promising results after a year or, say, four years — but by year nine, the results tend to look much less rosy, perhaps because the control group catches up with the treatment group. So it’s very helpful when economists do a 10-year follow-up to check whether the initial results persisted over time.

That’s why it’s worth drilling down into another randomized controlled study published by Duflo, Banerjee, and co-author Garima Sharma just a few months ago. This study followed up on an experiment conducted in West Bengal, India. Ultra-poor people were given two cows or two goats, together with training on how to generate income from the livestock and a small subsistence stipend to keep them going. The researchers found that the initial results persisted, with study subjects enjoying higher income and consumption even a full decade later.

Things are complicated: 

When [Shameran Abed, the executive director of BRAC] looks at all this evidence, he thinks the upshot is clear. “Graduation programs are much more impactful in the longer term,” he said. “I know that for the ultra-poor.”

But others, like Banerjee, are more circumspect. “I don’t think we can say that yet,” he told me. “I think it’s hard to read the evidence.”

Why? For one thing, while graduation programs appear to work great in some places, they’re dependent on the market — and they can run into problems in places where the market is either too dysfunctional or, ironically, too functional.

One randomized trial in India, published in 2012, is an example of the latter. It found that a graduation program yielded no net impact. Although it shifted participants away from agricultural jobs to other sorts of work, they could’ve earned just as much in their original agricultural jobs. While those original jobs were far from big money-makers, wages for agricultural labor had been improving in India, thanks to programs like the ambitious National Rural Employment Guarantee, so adding in a graduation program didn’t really help.

Dysfunctional markets produce their own obstacles. Abed told me about his experience trying to run a graduation program in Balochistan, an extremely dry, desert-like province in southwestern Pakistan, where participants were taught how to run a small business. One problem: There wasn’t a functional market for the businesses to thrive in. “Once they graduated, there wasn’t much to go to,” said Abed. “And there wasn’t microfinance available. So it was very, very difficult.”

Arguably, this points to an issue with the graduation approach’s reliance on the concept of the “poverty trap.” The idea here is that poverty works like gravity: to help someone escape it, you have to get them above a certain escape velocity. If you don’t give them a big push that gets them above that threshold, they’ll eventually sink back into poverty.

“I think the poverty trap concept is very simplistic,” said Miriam Laker-Oketta, a Uganda-based research director at GiveDirectly, which runs cash transfer programs. “It makes it look like you just need to get this one person out of poverty. But because they’re human beings, they’re all connected. The community is all connected. I think we need to be thinking of poverty in terms of communities rather than individuals. It has to be more systemic.”

1. ^{^}

   Cold take: many news outlets undervalue more continuous, longer-running-but-important stories, because they're not recent or unexpected enough. You can take your pick of lists of criteria for newsworthiness and see if you think this really tracks the importance of given events or phenomena. 

   Related: "Does the news reflect what we die from?" by Our World in Data. 

   

2. ^{^}

   Here's a potential source/relevant study — I haven't yet read it, but it looks interesting. 

Stephen Clare @ 2022-07-07T17:01 (+31)

[Epistemic status: Writing quickly about a complicated literature - I think the direction of this critique is right but would love to discuss further in the comments!]

I know authors don't choose their own headlines, but this one really is a ridiculous overstatement. We can't crown something the "best" way to help extremely poor people after only comparing two interventions (graduation vs. cash). And even in a head-to-head comparison, the available evidence suggests that the characteristics and effects of graduation-style programs vary so much across contexts that we should be extremely cautious about generalizing from a study of one particular program in one particular place.

At the risk of further beating an already dead horse, I also think it's worth re-contextualizing this article in the context of the growth vs. randomista debate (which, IMO, has not advanced much at all since Hauke and John's post in Jan. 2020. Here again is Pritchett's damning figure comparing the gains from a representative $1,000,000,000 investment in Graduation-type programs to the value of various national growth accelerations:

Overall I feel pretty disappointed at how un-quantitative the linked Vox article is. Sigal summarizes the results of  the latest BRAC study as "study subjects enjoying higher income and consumption even a full decade later." But this is just saying the program has AN effect. This is trivial. What matters is the size of the effect. Unfortunately, Sci-Hub hasn't indexed the 2021 study the Vox article focuses on. But I've looked at the results of previous graduation studies and haven't been blown away. Banerjee et al. (2016) report of BRAC that "seven years after the asset were first distributed, the monthly consumption of those assigned to treatment is 16 dollars–or 25%–higher [than those in the control group]".

Of course, 16 extra dollars per month works out to $0.50 per day. I believe in diminishing returns to consumption such that an extra 50 cents per day is very meaningful if it boosts your consumption by 25%. But any claims that Graduation helps people "graduate from" or "escape" poverty assumes a low poverty line, such as the standard $1.25 per day. That is, the average very poor Graduation program participant remains very poor after participating in the program. I think it's misleading to talk about them escaping poverty as Vox and Graduation proponents do: e.g. from the article "[BRAC] aims to “graduate” recipients out of extreme poverty."

I'm impressed by the thoroughness of Sigal's literature review here but still think it understates the extent to which these programs are controversial. In fact, I basically think we shouldn't generalize at all from studies of one particular Graduation program. Some of these issues are discussed in what I think is a pretty good critical article, Kidd & Athias 2019, which I don't see discussed in the Vox piece.

Lizka @ 2022-07-07T17:14 (+13)

I really appreciate this comment, thank you!

I agree with your disappointment about the lack of any quantitative aspect, and I'm adding the paper you linked to my reading list. 

I've also been planning on reading selected books and papers from Further reading/References in the Growth and the case against randomista development for a while, but if you have other recommendations, I'd love to hear them. 

Benjamin Stewart @ 2022-07-08T00:06 (+11)

Here's a link to the Banerjee paper for those without institutional access.

Stephen Clare @ 2022-07-08T13:08 (+5)

Thanks for the link. I want to emphasize that I think this is a very good paper. The intro especially is well worth reading for its description of the program and poverty trap model.

Here's a relevant quote; the results aren't much of an update as the absolute treatment effect in terms of per-capita consumption didn't change between years 7 and 10.

Average household per capita consumption was $1.35 (2018 PPP) at baseline in the control group and $2.90 by year ten. [...] [The treatment group's] per capita consumption is $0.60 per day (0.6 standard deviations) higher than the control group at both years seven and ten, and income is 0.3 standard deviations higher. This temporal pattern of growing effects followed by persistence is consistent with the alleviation of a poverty trap, what BRAC describes as the graduation of treated households. However, it is also consistent with there being persistent effects without actually getting out of a trap: the control households do become less poor over time, and the treatment households are still not very rich by the time the treatment effect stabilizes (although their average consumption is above the World Bank threshold for “moderate poverty”).

(p. 472, emphasis mine)

Note that the authors' wording is more cautious and nuanced than the Vox article.