Unjournal's evaluations of "Intergenerational Child Mortality Impacts of Deworming"

By david_reinstein @ 2024-10-25T20:36 (+11)

This is a linkpost to https://unjournal.pubpub.org/pub/evalsumintergendeworming/

The Unjournal organized two evaluations of the paper: “Intergenerational Child Mortality Impacts of Deworming: Experimental Evidence from Two Decades of the Kenya Life Panel Survey”.  The authors find evidence for an intergenerational mortality benefit that could be large enough to make a difference to the debate over the effectiveness of deworming (relative to other impactful interventions like bednets). 

See these evaluations, ratings, our summary, and the author's brief response in the package here.

See fairly non-technical discussions, for context

The evaluations

The evaluators generally found the paper credible, but raised some concerns (e.g., about consistency with the pre-analysis plan) and requested some robustness checks.

From the second anonymous evaluator:

The  analysis would be more credible if the authors were transparent about their deviations from the pre-analysis plan, which include the addition of (and exclusion of) outcomes and changes to the empirical specification.

E1 was also asked to identify & assess the paper's 'key claim'.

I believe with fairly high certainty the general claim that mortality of the children of those randomly exposed to the intervention fell and ... it likely falls within the [implied reported] confidence intervals.

(Note, we are working to encourage more specific claim identification and greater quantification of these beliefs and uncertainty. In this vein, please also see our Pivotal Questions trial.

The authors responded briefly, noting they are revising the paper in response to these and other comments. 

 

Our notes on 'why we prioritized this paper'

There is substantial debate over the effectiveness of deworming relative to other impactful interventions like bednets. Evidence for the effectiveness of deworming seems to be far more uncertain with much greater confidence intervals than for other interventions. Previous work, particularly that focusing on the long-term impact, seems to be inconsistent, or in some cases seemed face-value implausible. 

More evidence and more synthesis, and more careful scrutiny would seem to be high value; the magnitude of the estimates of the impact of deworming, and our confidence in these estimates seems pivotal to driving charitable funding and intervention recommendations.

The intergenerational mortality benefit seems substantial enough in magnitude to make a difference as confirmed by the paper authors, and by a very quick chat GPT-assisted back-of-the-envelope calculation and review. (See here, may contain errors, not checked.)