New intervention: paying farmers to not burn crops

By Karthik Tadepalli @ 2022-12-22T17:18 (+88)

• I summarize a recent paper evaluating an intervention to reduce air pollution: paying farmers not to burn their crop residue.
• A pure conditional contract is ineffective, but when farmers are paid some fraction upfront, they reduce crop burning significantly.
• The authors calculate that this contract saves a life for $5,000, comparable to GiveWell's top charities; however, these averted deaths may be among the elderly and thus not reflect many years of life saved.
• I also calculate promising climate co-benefits; this intervention reduces GHG emissions for roughly $36 a ton, which costs less than the social cost of carbon, but more than the best climate mitigation strategies.
• I strongly recommend further research and piloting as a way to build on this single study, especially given the lack of scalable air pollution interventions in EA.

Air pollution causes at least 7 million premature deaths each year. Despite this, it has only recently surfaced as a top cause for effective altruists, with Open Philanthropy announcing South Asian air quality as its newest focus area last year. Even with this recent focus, grants have focused mostly on research rather than on interventions to actually improve air quality. The problem is that we don't yet have shovel-ready air pollution interventions, demonstrably cost-effective interventions that are feasible and scalable for charities to implement themselves (rather than relying on uncertain advocacy). I think one recent paper offers a cost-effective, feasible and scalable intervention to reduce air pollution, and we should investigate it much more closely.

Overview

One pernicious source of air pollution in developing countries^[1] is crop residue burning, where farmers burn the remnants of their summer crop to quickly clear the fields for their winter crop. The pollution from this burning causes 66,000 premature deaths a year in India alone. Both bans on crop burning and subsidizing alternatives have failed, leaving crop burning in desperate need of a solution.

A new working paper by Kelsey Jack, Seema Jayachandran, Namrata Kala and Rohini Pande proposes one solution: paying farmers directly not to burn crop residue, with a partial payment upfront and the remainder conditional on not burning. They show this contract reduces crop burning with an RCT, while purely conditional payments have no effect. Importantly, they also calculate that this approach saves a life for around $5,000, which is competitive with GiveWell's top charities.

I think this study poses an important line of research for EA organizations to pursue, and I conclude with some ways for us to use this research.

The paper

One of the authors has summarized the paper already.

In brief: We use a randomized trial to evaluate a program that financially rewarded farmers if they avoided burning their rice stubble. We tried out a standard incentive contract that paid the farmer after we verified that he’d complied with the contract terms. That approach had no impact. However, when the contract was tweaked so that some of the payment was made upfront, the financial rewards program became a very cost-effective way to reduce burning, saving a life for <$5000. [author's emphasis]

I recommend reading her full summary, but here is a self-contained overview. The study took place in Punjab, India, where crop burning is an especially harmful practice. A breathtaking 10 out of the 11 most polluted cities in the world are in northern India, and much of their pollution comes from crop burning from Punjab and other northern states.

The study intervention is a form of payment for ecosystem services (PES) contract. PES contracts have been used and evaluated for other environmental goals (e.g. reducing deforestation by paying landowners and community members for forest land preserved), but are quite new as an intervention to reduce crop burning. In this context, a standard PES contract is made with a farmer at the beginning of the growing season and pays them a certain amount ($12 per acre) at the end if they do not burn their crop residue. In contrast, an upfront PES contract pays some fraction of this (either 25% or 50%) as an explicitly unconditional transfer up front, and the remainder at the end if the farmer does not burn their crop residue. The authors randomized farmers into three experimental groups:^[2] a standard PES group, an upfront PES group, and a control group which got no contract.

To measure the impact of these two contracts, the authors train a machine learning model on satellite images of fields to classify whether the field has been burnt. They use this model to detect crop burning in each group. They find that standard PES has no effect, while upfront PES makes a farmer 12 percentage points less likely to burn any field. For comparison, 90% of control farmers burnt a field and 78% of upfront PES farmers burnt a field, so the vast majority of farmers are still engaged in crop burning but the change is meaningful.

The authors explore a few potential explanations for why paying some money upfront makes such a difference. They have no conclusive evidence for any story, but two explanations have some support in their data:

1. Cash constraints - 70% of farmers said that cash constraints were important in their decisions of how to manage crop residue, so it's plausible that paying people upfront allowed them to pursue more expensive ways of disposing of crop residue. However, this didn't differ by treatment group.
2. Trust - upfront PES farmers were 7 percentage points more likely to trust that they would get paid. However, the control mean was 85% of farmers trusting they would be paid, so this can't explain that much of the effect.

Finally, some farmers argue that crop burning is necessary to prevent delays to winter cropping. However, the authors detect no significant effect of each contract on either winter yields or winter crop times.^[3] This is likely because farmers adopt other ways of removing crop residue; upfront PES farmers were more likely to buy a baler, which is used to remove residue from the field.

Cost-effectiveness

One unique feature of this paper is that it explicitly considers the cost-effectiveness of its proposed intervention, with a simple approach:

1. The authors use the treatment effect of upfront PES, and the cost of farmer payments, to calculate the cost per reduced acre of burning as being $37 per acre with the upfront PES contracts.^[4]
2. They assume 50% of rice acreage in three Indian states (Punjab, Haryana, Uttar Pradesh) is burned, based on this paper, translating to 8.8 million acres of burning.
3. They take the Global Burden of Disease estimate that crop burning caused 66,000 premature deaths in India in 2015, and thus each acre of burning averted reduces death by $66000/8800000=0.0075$ on average.
4. $37 to reduce deaths by 0.0075 implies that one death is averted for 37/0.0075 = $4,933.

Thus, they conclude that they save a life for around $5,000. For comparison, GiveWell calculates that the Against Malaria Foundation saves a life for $3,800 to $8,500 across countries. So this estimate makes PES for crop burning competitive with GiveWell's top charities.^[5]

A very important caveat with these results is that averting premature deaths is not the same as saving lives. Averting the death of an elderly person saves fewer years of life than saving a child. The mortality burden of air pollution is largely experienced by elderly people. One way to solve this issue is with the DALY framework (disability-adjusted life years, which count the years of life lost to both death and disability). This post previously included a cost-effectiveness analysis based on the DALY framework, but I've removed it.^[6]

Furthermore, two other caveats would increase the cost per life saved:

1. The authors have two methods of estimating reduced crop burning from satellite data^[7] and the estimate of $37 per acre comes from their preferred method. The other method gives a smaller effect and a 1.5x higher cost per acre burned, i.e. $55 per acre of reduced burning. This adjustment increases the cost per life saved to $7,333 using the authors' specification or $2,260 by my specification.
2. This cost-effectiveness calculation only uses the cost of payments as an input, rather than any other costs of delivery (e.g. surveyors to verify non-burning, fees associated with writing and enforcing contracts). The authors do not report the costs of delivery which would definitely raise the cost per life saved.^[8]

Climate co-benefits

While air pollution is likely the biggest source of harms from crop burning, crop burning also has a clear carbon footprint that the upfront PES contract helps reduce. One study estimates that crop burning in India is responsible for 9.1 million tons of CO2-equivalent GHGs^[9] annually. With 8.8 millions of acres burned, this means each acre unburned was responsible for 1.034 tons of GHGs averted. At a cost of $37 per unburned acre, upfront PES contracts offset 1 ton of GHGs for $36. This is a lower cost than all reasonable estimates of the social cost of carbon (the damage to society from 1 additional ton of GHGs). Thus, even if we ignore the air pollution benefits, the social benefits from PES contracts likely exceed their costs.

However, they are not the cheapest way to offset GHGs, and this estimate is still within an order of magnitude of reasonable social cost of carbon estimates, so it's possible that the pure climate benefits are less than the intervention's costs. As a result, the climate motivation for PES contracts on crop burning is weaker than the air pollution motivation.

Nonetheless, even with these caveats, the climate impacts are a clear co-benefit; they might not be enough to make PES contracts for crop burning a cost-effective intervention, but they make PES contracts look even better than the already-impressive picture from air pollution benefits.

Warning: wild speculation. These co-benefits could provide an appealing on-ramp for organizations to use the huge pool of carbon offset money to fund PES contracts for crop burning. Carbon offsets have faced serious scrutiny about their impact. If there is a lot of "dumb money" that organizations want to move towards more effective solutions, then PES contracts for crop burning could offer one such solution.

How would this scale?

Programs that work in an RCT need not succeed when scaled up by a government or charity. The policy implications of this study depend on how well its results would scale. The study can't answer this scaling question, but my best guess is that scaling would increase the efficacy of upfront PES:

1. Legal enforcement capacity: currently, crop burning is illegal, but this ban is unenforceable because of how widespread the practice is. However, if PES contracts were scaled up and more farmers adopted them, it would become much more feasible to punish farmers who do burn crops. This in turn increases the deterrent effects of the ban, creating a virtuous cycle that reduces crop burning.
2. Demonstration of feasible alternatives: farmers are aware of the harms of crop burning, but do it because they see no other way to preserve their winter crop. However, the study results suggest that some upfront PES farmers adopted alternative crop residue management methods without reducing winter crop yields. This implies farmers could learn more about these methods from implementing PES at scale, and that learning could reduce their crop burning organically.
3. Increased trust: the authors have suggestive evidence that an important failure mode for standard PES contracts is that farmers don't trust they will actually get paid at the end of the contract period. This is a natural concern for a new policy, but if an organization built an established reputation for abiding by its contracts, that could make farmers trust them more and thus be more willing to enter and stick to PES contracts.

However, one countervailing force reduces the likely cost-effectiveness of this policy at scale: the study districts were selected to have more crop burning than average. The study districts were chosen explicitly for their high rates of crop burning and low rates of alternative crop residue management methods. Thus, implementing PES contracts beyond their 300 study villages might have smaller effects if crop burning is less of a problem elsewhere.

Nonetheless, I don't think this selection issue is critical. Millions of acres of farmland are burned each year, and the farmers in this study collectively own a measly 8,000 acres. I think a PES program would have to be huge before it reached a scale that would seriously diminish the study's effects.

What do we do with this?

Let's say you accept this study's results for now. What can we do with this information?

The first takeaway is that PES is uniquely feasible for nonprofits to implement since it involves no regulation but rather a voluntary contract. Compare this to air pollution interventions that are only implementable by governments, e.g. vehicular efficiency standards - a nonprofit can only make progress by lobbying the government to enact standards, and success in that lobbying is uncertain. Thus, PES contracts are not just an academic proposal but a feasible and shovel-ready intervention that charities could scale up.

Second, we should do more groundwork to test this study's results. We shouldn't make important policy or allocate large sums of money based one study. Results can be p-hacked, false positives by random chance, fail to generalize to other places, or suffer from scaling issues. But the enormous cost-effectiveness of this intervention from a shallow investigation makes it a strong candidate for further research. Some very useful research to follow up on this would be:

1. Pure replication. This would not be in any researcher's incentives, but any organization that might consider implementing upfront PES as an intervention should do a pure replication of this study in their context. A neat feature of this study is that its measurement strategy uses commercially available satellite data, rather than confidential administrative data of any kind, so it can actually be replicated in any new context.
2. A replication with more power to detect mechanisms. The study sample is too small to really dig into why upfront PES works so much better than standard PES, or why standard PES fails. A larger study could show more heterogeneous treatment effects: for example, if farmers with more assets were less responsive to upfront PES than farmers with fewer assets, we could conclude that upfront PES helps by alleviating cash constraints for farmers to invest in non-burning approaches. The authors are unable to detect such an effect, but we can't say if that's because it doesn't exist or because of sample size issues.

Moreover, the climate co-benefits of PES contracts to reduce crop burning are a clear source of additional value. They make the case for PES contracts more robust to multiple priorities or value judgments. I also think they can provide a useful platform to access the large pool of carbon offset money: for climate-oriented funders, PES contracts for crop burning could be a generally effective way to reduce emissions while offering fantastic health co-benefits. From a practical perspective, this is quite important to keep in mind for funding sources if PES contracts for crop burning are found to be worth scaling up.

In short, PES contracts are an exciting and shovel-ready intervention to reduce air pollution, an area with few demonstrably cost-effective interventions for charities to implement. At first glance, upfront PES contracts for crop burning beat GiveWell's top charities in their cost to save a life. I'm sure that upon further scrutiny, they will lose some of that luster, but they merit a lot of attention. We should fund more research into upfront PES, and a real charity should pilot it, so that we can test one of the most promising air pollution interventions yet.

Thanks to Oliver Kim and Tejas Subramaniam for helpful feedback.