Sequencing Swabs

By Jeff Kaufman @ 2024-02-01T01:54 (+24)

This is a crosspost, probably from LessWrong. Try viewing it there.

null
Jeff Kaufman @ 2024-02-27T01:32 (+4)

The original version of this post had results from a simulation where the key results were off by a factor of 100. See the update at the top of the post for more.

JoshuaBlake @ 2024-02-04T11:02 (+4)

Thank you for this write-up, very interesting. I'm excited to see more investigations of different surveillance systems' potential.

Hopefully, the SIREN 2.0 study, running this winter, will generate some more data to answer this question.

A few questions now I've had time to consider this post a bit more. Apologies, if these are very basic, I'm pretty unfamiliar with metagenomics.

First, how do you relate relative abundance to detection probability? I would have thought the total number of reads of the pathogen of interest also matters. That is, if you tested the entire population you would have some reads on every pathogen even if the relative abundance of some pathogens is very low.

Relatedly, does the cost of the sequencing scale roughly linearly with the relative abundance required? That is, if your 40,000x figure is correct, would that imply swabbing is ~40,000x cheaper than wastewater?

Finally, could you please expand on your figure of percentiles vs relative abundance? Why does the number of swabs affect the relative abundance? If you double the number of swabs, I would expect that both the total reads and the number of SARS-COV-2 reads double, hence holding the relative abundance constant. Perhaps it's that the variance in the number of individuals infected and their viral load increases but the mean of all the lines is the same, is that correct?

And now one point of disagreement.

If you could cost-effectively swab a large and diverse group of people, this would allow surveillance with much lower sequencing costs than wastewater. But that's a big "if": swabbing cost goes up in proportion to the number of people, and it's hard to avoid drawing from a correlated subgroup.

I don't think this is that big an "if".

1% incidence per week is extremely high. For context, SARS-CoV-2 at 1% incidence per week implies a prevalence of 2-3%. The UK, which had a middle-of-the-road rich world pandemic, peaked at this prevalence just before implementing a lockdown in the January 2021 (Alpha variant) wave.

The main scenario discussed (in the context of NAO) is stealth respiratory pathogens. Respiratory pathogens spread fairly indiscriminately because they spread in public areas. This is unlike pathogens spread through close contact (eg: HIV or mpox), which can be contained in smaller communities more easily.

In other words, a respiratory pathogen at 1% incidence per week is as widespread as SARS-CoV-2 at its worst in the pre-vaccine era, and spreading roughly as indiscriminately (it could be spreading slower). I don't see it being contained in hard-to-target populations.

Furthermore, wastewater sequencing doesn't fully solve this. Unless you cover all wastewater across the globe, you're still not covering everything. If you do large cities it's probably not as bad as individual swabbing but I'm not sure it's much better. Other settings, such as airports/airplanes, seem at least as bad as the populations you might target for convenient swabbing (eg: hospital or primary care patients).

So while the sample might be a few times either way biased, I'm sceptical this is bridging a 40,000x gap (maybe 40,000x isn't the relevant benchmark here - see comments previously).

Jeff Kaufman @ 2024-02-04T12:52 (+4)

Lots of great questions!

the SIREN 2.0 study, running this winter, will generate some more data to answer this question.

Thanks for pointing this out; I hadn't seen it and it's super relevant. I don't see what sample type they're using in the press release, but any kind of ongoing metagenomics to look at respiratory viruses is great!

how do you relate relative abundance to detection probability? I would have thought the total number of reads of the pathogen of interest also matters.

It depends on your detection method, but modeling it as needing some number of cumulative reads hitting the pathogen is a good first approximation.

If you think it would take N reads of the pathogen to flag it then if you know RA(1%) and the exponential growth rate you can make a back of the envelope estimate of how much sequencing you'd need on an ongoing basis to flag it before X% of people had ever been sick. For example, if you need 100 reads to flag, it doubles weekly, and RAi(1%) is 1e-7 then to flag at a cumulative incidence of 1% (and current weekly incidence of 0.5%) you'd need 100/1e-7 = 1e9 reads a week.

(I chose 1% cumulative incidence and weekly doubling to make the mental math easier. At 1% CI half the people got sick this week and half in previous weeks, and the cumulative infection rate across all past sequencing should sum to 1%, so we can use RAi(1%) directly. Though I might have messed this up since I'm doing it in my head lying in bed.)

if you tested the entire population you would have some reads on every pathogen even if the relative abundance of some pathogens is very low.

If you collected a large enough sample volume and sequenced deeply though, yes.

Relatedly, does the cost of the sequencing scale roughly linearly with the relative abundance required? That is, if your 40,000x figure is correct, would that imply swabbing is ~40,000x cheaper than wastewater?

It doesn't, for three reasons:

  • Sequencing in bulk is a lot cheaper per read. You might pay $13k for 10B read pairs, or $1k for 100M. But that's just ~10x.

  • Some components (lab time, kits) vary in proportion to the number of samples and don't go up much as your samples are bigger.

  • It's only your sequencing costs that vary with relative abundance, and while with wastewater I expect the cost of sequencing to dominate that's not the case for any other sample type I can think of (maybe air?) If you're sampling from individuals the cost of getting the samples is likely quite high (we were recently quoted $80/person from a contractor, and while I think we can do better if you want 1k people per pooled sample it's almost certainly more expensive than the sequencing charge).

Why does the number of swabs affect the relative abundance? If you double the number of swabs, I would expect that both the total reads and the number of SARS-COV-2 reads double, hence holding the relative abundance constant.

Some people have vastly higher viral loads than others, and the relative abundance you see for a pool depends on whether you get some of these people. Your intuition would be correct for pools large enough that this variation was no longer relevant.

I don't see it being contained in hard-to-target populations.

Sorry, I was unclear! The easiest way to collect a pooled sample is the walk around some building and sample everyone. This gets you a big sample pretty cheaply, but it's not a great one if you want to understand the containing city because it's likely that many people in the building will get sick on a similar timeframe. The sample members are too correlated in their exposure.

In your UK example, I'm guessing you could sample some office buildings of 1k people and find 0 cases and others and find 200 cases.

To avoid this you need broader sample collection, but that's logistically more difficult and so more expensive.

Airport arrivals would be great, though that's a difficult setting to work in.

I'm sceptical this is bridging a 40,000x gap (maybe 40,000x isn't the relevant benchmark here - see comments previously).

I'm also skeptical! I think sampling from individuals is extremely promising. It seems like you ought to be able to get down to more like $2/person in which case a pool of 1k costs you $2k in collection. Then add in $1k for sequencing and you're still well above wastewater. But my initial attempts to partner with people already doing sampling haven't turned up good leads.

JoshuaBlake @ 2024-02-08T17:46 (+4)

Thank you for that very detailed reply Jeff, I learnt a lot about how to think about costing this.

The easiest way to collect a pooled sample is the walk around some building and sample everyone. This gets you a big sample pretty cheaply, but it's not a great one if you want to understand the containing city because it's likely that many people in the building will get sick on a similar timeframe.

I agree this is true for an office block, but I would think you can do much better without much cost. For example, if you use a high-traffic commuter train station or supermarket I would guess you get a fairly broad cross-section of the city. They'd be somewhat uncorrelated (different home locations with children at different schools, different offices etc.) although obviously the geographical component is still there. Perhaps similar to wastewater though? You could do multiple locations as well though.

It seems like you ought to be able to get down to more like $2/person in which case a pool of 1k costs you $2k in collection. Then add in $1k for sequencing and you're still well above wastewater.

These numbers are maybe optimistic, but not ridiculously so.

The Coronavirus Infection Survey (big UK study which I've worked on) cost ~£1b for ~11.5m swabs (Excel sheets with data from Mar 2023 and historical data). Works out as ~$100 / swab.

Very likey overestimated upper bound though because that is a proper random sample of the whole population, with ~9.5m of the swabs collected by study workers going to houses. I think this budget might exclude the cost of PCR testing (done individually, not pooled) and a lot of time spent running / analysing the data.

Jeff Kaufman @ 2024-02-09T22:58 (+4)

if you use a high-traffic commuter train station or supermarket I would guess you get a fairly broad cross-section of the city

Definitely! Right after writing to you I started thinking about this, estimating costs, and talking to coworkers; sorry for not posting back! I do think something along these lines could work well.

These numbers are maybe optimistic, but not ridiculously so.

My main update since then is that if you do it at a transit station you probably need to compensate people, but also that a small amount of compensation doesn't sink this. Giving people $5 or a candy bar for a swab is possible, and if a team of two people at a busy transit station can get 50-200 swabs in an hour that's your biggest sample acquisition cost. I still think $1k is practical for the sequencing.

I'm trying to come up with examples of people doing something similar, which we'd want for presenting this to the IRB. Two examples so far:

Do you know of anything else that feels similar to this? People in public areas collecting biological samples from volunteers (perhaps lightly compensated).

JoshuaBlake @ 2024-02-11T10:46 (+4)

Do you know of anything else that feels similar to this? People in public areas collecting biological samples from volunteers (perhaps lightly compensated).

Afraid not. The closest I can think of is collecting samples from healthy volunteers without any benefit to them, but not in public areas. In particular, I'm thinking of swabbing in primary health settings (eg RGCP/UKHSA run something like this in England, I can't remember if it only includes those with respiratory symptoms) and testing blood donations (normally serological testing looking for antibodies). REACT (run by Imperial College) did swabbing for COVID via postal recruitment.

A bit of an aside, so maybe not of interest, however, this made me think of serological testing of residual blood samples. That is, when blood is collected for testing (for any clinical reason), not all of it is used in the tests, and the remaining (residual) part is tested. Here, there are no sample collection costs (the blood was collected anyway). However, it doesn't map exactly because you don't swab people without respiratory suspicion but you might take blood (eg anemia). Maybe there is an opportunity for either testing blood samples for pathogens (but I have no idea what that looks like) or samples taken for other respiratory reasons (but then you need to think about co-infection, ie does infection with influenza make you less likely to have another respiratory infection).

Finally, some shameless self-promotion. I'm currently nearing PhD competition with nothing lined up. If there are projects looking at these sorts of questions interested in modelling / stats / epidemiology input I'd be very interested, please DM. Please ignore this if unappreciated.

JoshuaBlake @ 2024-02-02T15:56 (+4)

Feel free to message me if you're interested in going deeper into what a typical viral load might look like. I can generate trajectories, based on the data from the ATACCC study. Note that this is in viral RNA copies, not Ct values - they did the conversion as part of that study.

Jeff Kaufman @ 2024-02-02T16:02 (+2)

Thanks! I'm most interested in viral load in the sense of the relative abundance you get with untargeted shotgun sequencing (since you need sequencing (or something similarly general) to detect novel threats and/or avoid having a trivially-bypassable detection system) but there's not much literature on this.