Lant Pritchett on the futility of "smart buys" in developing-world education

By Aaron Gertler 🔸 @ 2021-07-18T23:00 (+58)

Why I wrote this post

Education isn't something many people explicitly focus on within EA. But I still think it's good to highlight strong writing on a variety of cause areas for a few reasons:

• Reading about the real world helps you build better models of it.
• Reading about difficulties with one cause area might help you anticipate similar difficulties in other areas (some of the discussion below could easily apply to health interventions, institutional decision-making, etc.)
• Lots of people ask why EA isn't very focused on education (or any number of other causes). It's useful to know why some areas are more promising so that we can explain why we prioritize certain things.

I thought Pritchett's writing below (on primary and secondary education in the developing world) was well-written and reasonable (albeit rough, as it consists of an informal blog post and two early-draft PDFs). I learned a lot about developing-world education; a few new ideas, and lots of building on "things I thought I knew" with actual evidence.

If you're interested in education as a cause area, or just as an "interesting thing to think about", I definitely recommend Pritchett's blog post + short paper. If you like those, the long paper is also worth a skim.

The problem with "smart buys" in education

The Copenhagen Consensus hires economists to write lists of "smart buys" — things a donor could fund that would have some impressive impact for developing countries.

In 2008, they asked Lant Pritchett to make one of these lists for education. He sent back a paper without one, and gave back what he'd been paid. He thought such a list would be futile, given that:

• Promising education RCTs seldom worked at scale
• Much discussion of education policy was fundamentally confused — the goals, incentives, and theories of development economists didn't match those of local governments, teachers, or students.
  • Thus, recommending ways for governments to maximize an "education output" through their spending wouldn't make sense, as they clearly weren't looking to do this in the first place.
  • If spending were to have an impact, certain factors would have to be in place — factors that were often rare in the developing world. (For example, reducing school fees wouldn't do much unless schools were actually effective at educating students.)

He draws the following analogy:

Suppose I have just read that spinach and broccoli are “cost-effective” foods in providing high nutritional content at low prices. I am in the grocery store and see a fellow shopper whose cart is loaded with food that is both bad for you and expensive (e.g. sugared breakfast cereals) and nothing really nutritious. I could then go up to her/him and make a “recommendation” and give him/her my empirical evidence grounded “smart buy” advice: “Hello stranger, you should buy some broccoli because it is a cost effective source of vitamins.” 

One can imagine many outcomes from this but perhaps the least plausible response is: “Thanks, fellow stranger, I will now buy some broccoli and integrate this cost-effective source of vitamins into my regular food consumption habits.”

Take the analogy a step further and suppose I have an altruistic interest in the health of my fellow shopper and so I just buy and broccoli and spinach and put it into his/her shopping bags for free. Again, one can imagine many outcomes from this action of mine, but I would think the most probable is that some broccoli and spinach gets thrown away.

This doesn't mean there aren't good ways to invest in better education — just that RCT evidence alone is a poor substitute for understanding incentives and structures on the relevant local/national level, and that those factors will doom some naïve approaches.

In the end, Pritchett wrote two papers for the Copenhagen Consensus:

• "The Challenge of the Lack of Education" sums up a huge amount of evidence on schooling practices, and ends with a set of potential areas for improvement — no "smart buys" with estimated impact, but some general approaches that might work if certain factors are in place.
• His perspective paper on education, a reply to "smart buy" recommendations from other economists, explains some key difficulties in trying to "buy" more education. It's much shorter, so I'd recommend starting here if you feel like spending some time in the head of a smart, cantankerous researcher.
   

Excerpts from the short paper

There are very large private returns to schooling—on average, across the 42 developing countries [the economists examine], an additional year of schooling is associated with an 8.2 percent increase in wages for men and a 10.3 percent increase in wages for women, and a 9.2 percent increase in urban areas and 8 percent in rural areas. 

Why does this make [the economists'] life difficult? Because if I want to justify public spending on something, I want the private gains to be small and the public gains to be large. But what they show is that the private gains are considerable, which means that, if one were to justify public spending as a large proportion of the private spending, one needs very large externalities to education. 

The most common reason given why children are not attending school is “lack of interest”—which is 47.3 percent of urban children and 44 percent of rural children not in school. 

Why do these children “lack interest”? This is hugely important to understand, as it also influences how we understand the other responses. If someone says the reason a child is not in school is because they “work outside the home” that may just be begging the question as the question is “why do they work outside the home and not go to school?” 

While it is obvious that children not in school work more, it is not so obvious how much is cause (children drop-out of school in order to work) and how much is effect (once children have dropped out (for other reasons) they work more). 

Here is my conjecture. Going to school reveals two things. First, it reveals your adeptness for formal schooling (not some catch all like “intelligence” but just how good at school you are). Second, it reveals the quality of your school. By the time most children reach, say 14 or 15 years old, many of them “lack interest” in schooling because either (a) they have realized they are not adept at schooling (and hence do not like it) or (b) they realize the school they are in is miserable and/or no learning is going on or both (a) and (b). 

Rather than the model of parents pulling children out of school to work (in the market or at home) I would suspect the much more common phenomena is children pleading with their parents to not have to go to school. 

Just as an example, Tyack (1974) tells that Helen Todd, a factory inspector in Chicago interviewed 500 children working in factories (often in dangerous and unpleasant conditions) and asked the question: “If your father had a good job and you didn’t have to work, which would you rather do—go to school or work in a factory?” Of those 500, fully 412 said they would choose factory work. She recounts asking one fourteen-year-old girl in one particularly unpleasant factory (lacquering canes, involving heat and turpentine) why they did not go to school and got the response “School is the fiercest thing you can come up against. Factories ain’t no cinch, but schools is worst.”

The results from Progresa—of impacts on attendance but not on measured learning outcomes—is consistent with the view that marginal students were forced back into marginal schools. 

According to [the economists], iron supplements to secondary school students has a BCR [benefit-to-cost ratio] of 32. What is the market failure? One can just buy iron supplements on the market right? So if parents knew of these private benefits from a private action, then they should willingly adopt it, right? 

So [instead of] a policy of “paying for iron supplements” why not a policy of “publicizing the benefits of iron supplements”, from which one should get much higher uptake with less cost? 

Similarly with something like Balsakhi tutoring. There is a huge flourishing market in India for private schools and tutoring. If you really believe the BCR is 528—you can get 5,152 in benefits for only 9 dollars—why [not simply make] parents aware of the potentially massive benefits of a particular type of tutoring? The behavioral model is that parents will turn down a low total cost, [hugely beneficial] intervention? 

If not, then spreading the information (the creation of which is a public good) should be enormously more cost effective as a policy than scaling up the program (or alternatively the scaling up of this program should be an enormous cash cow the private sector would willingly scale up).

Excerpt from the long paper

This paper goes into lots of detail on specific interventions in a way that felt hard to excerpt from (the value is in the details!). But I did think this was well said:

Since many of the readers of this document will have no direct experience living or working in the poorer countries of the world in which the lack of education is most pressing I would ask them to abandon now any assumptions based on their personal experience. 

Citizens of the developing world cannot take for granted what every resident of a rich industrial country does. In particular, many of the same words are used in discussions—but they convey problems of different orders of magnitude:

• When “poor pedagogical approaches” are discussed, it is usually taken for granted the children are not being hit with by the teacher with a stick—not always so.
• When “under motivated” teachers are discussed, it is usually assumed they at least show up—not always so.
• When “poor facilities” are discussed, it is usually assumed there is at least a classroom—not always so.
• When “political patronage” is discussed, it is usually assumed that teaching positions are not openly sold as patronage—not always so.
• “Low quality teachers” is not assumed to not mean “cannot read and write at a fourth grade level”—not always so.