Sharing insights from my master’s work on the Global Health Security Index: seeking feedback and research directions

By Vincent Niger🔸 @ 2024-11-25T12:06 (+45)

This is my first post on the EA Forum. (Yeah, I'm nervous.)
 

Note: The Global Health Security Index (GHSI) is the first comprehensive assessment of health security capabilities across 195 countries. Created by the Nuclear Threat Initiative (NTI), the Johns Hopkins Center for Health Security, and The Economist Intelligence Unit, it evaluates each country's ability to prevent, detect, and respond to health emergencies using publicly available data across six categories: prevention, detection, rapid response, health system, compliance with international norms, and risk environment.

About me

I’m Vincent Niger—nurse, Europubhealth+ Erasmus Mundus double master’s candidate in Public Health, currently based in Maastricht, where I’m enrolled in the master’s in Governance and Leadership in European Public Health. I’m also a facilitator for BlueDot Impact's Pandemic course, which I cannot recommend warmly enough. Seriously, check it out! I’m exploring the possibility of shifting my career toward biosecurity and pandemic preparedness.

The research

I’ve been focusing my master’s assignments on pandemic preparedness. For one of my courses, Measuring and Comparing Health in Europe – Quantitative and Qualitative Approaches, I wrote a paper titled 'Assessing the Predictive Power of the Global Health Security Index: Correlation with COVID-19 Excess Deaths in EU Countries.' Over the four weeks I spent on this research project, I worked within the course's rubric, which included these four aims (in case you’re wondering about the somewhat odd framing):

1. Cross-country benchmarking based on the target indicator.
2. Critical appraisal of the target indicator.
3. A quantitative or qualitative exploration of at least one factor influencing the target indicator.
4. Reflection on the probability of a causal link between this factor and the indicator.

You can access the paper here. Below is a concise summary by my friend Claude:

This study examined how well the Global Health Security Index (GHSI) predicted COVID-19 outcomes in European countries by analyzing the correlation between countries' 2019 GHSI scores and their excess death rates during the pandemic. Key findings:

1. The GHSI showed only moderate correlation (-0.38) with excess deaths, though this improved significantly (-0.76) when excluding very small countries.
2. Most interestingly, of all GHSI criteria, the "Risk Environment" category (Criterion 6) was the strongest predictor of excess deaths, suggesting that general country vulnerability may matter more than specific health security measures.
3. The study found that similarly-ranked countries (like Spain, France, Germany, and Finland) had vastly different death rates, raising questions about the index's predictive power.

Recommendations for improving the GHSI include:

• Giving more weight to risk environment factors
• Incorporating crowd forecasting methods
• Better accounting for contextual factors like political stability and public trust
• Regular validation against real outcomes

The paper concludes that while the GHSI is valuable for identifying gaps in health security, it needs refinement to better predict actual pandemic outcomes.

Reasons for sharing here

• It’s always good to put your ideas out into the community, even when they’re still in development. Facing the anxiety of potential negative feedback is a growth opportunity, right?
• I think some of the claims I make in this assignment are on shaky ground, and I’d really value your thoughts on them (more on this below).
• I’m also considering diving deeper into the GHSI for my master’s thesis and would greatly appreciate guidance (more on this below).

Self-criticism and questions

• The statistical analysis in my paper is very basic. If anyone here could suggest more advanced tools or approaches, I’m all ears!
• Spearman's rho for the correlation between excess deaths and Criterion 6 (overall risk environment), and between excess deaths and the GHSI score (when 100% weight is put on Criterion 6), showed different results. I can’t figure out why, except that I made a mistake at some point handling data. Is there another potential reason?
• The literature review was, um, a bit rushed (let’s call it “time-constrained,” shall we?). If you’ve come across interesting papers on the utility or limitations of the GHSI, please share them!
• My claim that crowd forecasting could improve the GHSI’s predictive value is based less on expertise and more on vibes I’ve absorbed from the EA community. Is this a good idea? If so, how might it work in practice? I’d love input from those with experience in this area.

Key Findings

Here’s the main takeaway from my paper:

1. My analysis (and other research) suggests the GHSI findings only moderately correlate with COVID-19 outcomes, at least when looking at excess deaths.
2. We don’t know how well the GHSI findings will correlate with outcomes in future pandemics—or whether they’d be more predictive for other types of pandemics.
3. There’s potential to improve the GHSI’s predictive power by exploring options like crowd forecasting, refining criteria and weighting, etc.
4. But the big, burning question for me is this: 

How effective is the GHSI at actually driving change in the biosecurity field?

Looking Ahead

On the GHSI’s website, it says:

“The GHS Index aims to spur measurable changes in national health security and improve international capability to address one of the world’s most omnipresent risks: infectious disease outbreaks that can lead to international epidemics and pandemics.”

To my knowledge, no research has assessed how well the GHSI achieves this goal. For my master’s thesis (which must focus on Europe), I’d be excited to explore questions like:

• How do stakeholders in the European biosecurity field engage with the GHSI’s findings?
• How do they interpret these findings?
• What actions, if any, are driven by them?

Preliminary chats with NTI Bio’s team suggest these are interesting research avenues. But since I’m pretty new to research, quantitative, but especially qualitative methods, I’d deeply appreciate insights from experienced researchers. Specifically:

• What methods could I use to explore these questions?
• How might I refine my research question to make it both impactful and feasible within a 10-week placement?

Does this sound like I’m asking you to write my thesis for me? Nah, I promise I’m not. I just know this community has great minds, and I’d love to tap into your collective wisdom.

Also, if you’d like to chat more about this or other topics, I’m always up for a call. Let me know! 😊I'm on LinkedIn, and EA Anywhere's Slack, amongst others.