The real false positive rate is much higher than the 5% reported.
The false positive rate reported is ~5% in a sample with an equal number of people that did and did not develop cancer, while the ratio in reality is 1/122 (i.e., 1000 people developed cancer, 122000 did not).
Could still be a useful development, but it does make me wonder about the cost/benefits at this stage.
Yes, the positive predictive value of the test may be low, with the PPV being equal to the number of true positives divide by the number of true positives plus the number of false positives.
Since this test is designed to identify cancer very early, a likely solution to the low PPV problem would be to re-test the individual, perhaps even multiple times. The likelihood of multiple consecutive FP test results, barring a biological reason for the FP, would at some point warrant a more accurate test, be it a screen or diagnostic test depending on tumor type. Also keep in mind that the PPV of the test would increase with a person's age - an 80 year old is at a greater risk of developing cancer than is a 30 year old. This test would also be useful for people with hereditary cancer syndromes.
The cost benefit would be huge - imagine if everyone could identify their cancer 6 months or even years earlier. Mortality would greatly decrease, as would spending on treatments, etc.
I think that's an unwarranted assumption. Sure, another source of false positives is contamination or user error even, but other than that most FP are indeed due to a (usually unknown) biological reason. I'm reminded of a friend that was just positive she was not pregnant. She tested herself three times with three different pee stick brands. She was wrong. They all use the same chemistry, and for some reason something in her body interferes with those tests.
The cost benefit for someone with a cancer diagnosis could be huge. But for the people with false positives, that could also be very harmful to people. A (false) cancer diagnosis would be incredibly stressful.
This seems like a failure in how test results are communicated to patients. Patients should just been told that they have been "flagged for further testing", rather than being told they have tested positive for cancer.
So I have a real issue with not just telling the patient flat out that their test results indicate a high risk of having cancer and they need to be tested again.
As a society, we're just getting too used to accommodating people's ignorance about statistics and science. Where does it end?
Do we have to really have to give up promising therapeutics and diagnostic tests because we're afraid people's feelings will be hurt?
Yeah, I really hate it when people say what sounds like, "We recommend against even trying this test because people are incapable of responding appropriately to the answer."
It shouldn't even be that hard to state. "X% of people your age [+ whatever other factors they're using for a reference class] have [type of] cancer. Based on that and your test results, there's a Y% chance you have that type of cancer." Forgive me if I'm too disconnected from the average person, but is that too difficult to understand?
> So I have a real issue with not just telling the patient flat out that their test results indicate a high risk of having cancer and they need to be tested again.
The problem is that "tested again" has its own complications.
This is not theoretical. Part of the reason for updating the breast cancer screening guidelines is the fact that biopsies to confirm were causing more problems than the "cancers" they were finding.
You mean that the test physically caused a disorder in the patient because of a side effect, or that there was a problem in the patient’s attitude or emotional response? Because those are very different failure modes.
In one case the patient’s attitude about the test is not the determining consideration. A perfectly stoic and rational patient would be just as vulnerable to the side effects as anyone else.
A biopsy causes complications up to and including the death of the patient (secondary infections, etc.).
A biopsy is the next step to confirm "cancerous/non-cancerous" after a mammogram screen gives a "positive" result.
The problems with the screening is that once a "positive" comes up on the screen most patients cannot stop themselves from going to the next stage even if the probabilities don't warrant it.
The only medical "solution" was to decrease the frequency of screenings and raise the age at which first screening should occur.
Hmm, is that a situation where there is no appropriate action to take, regardless of the outcome of the screen? If that were the case, then there would be no point in doing the screen even if all actors were fully rational. But I'm guessing it's more like, the correct action is to wait N months and do a followup test.
Does it not work to say "If we take a biopsy, assuming normal treatment plans and outcomes for someone in your reference class, there's a 0.1% chance the biopsy kills you, and a 0.05% chance it saves you from cancer compared to the plan I'm recommending", or whatever the truth is?
(If the next possible step is a biopsy, under what circumstances does the calculus imply one should do the biopsy? When the screen results look sufficiently bad? Then maybe they shouldn't be presented as a binary "positive"/"negative".)
A diagnosis of "you might have cancer but the probability is low and we should just watch it" is not something that most people can psychologically take.
For more than half of the population, that kind of understanding is simply untenable.
And, even for people I knew who could understand it, several of them had so much anxiety that it drove their blood pressure up and it was better to just go get the biopsy so they didn't wind up with a stroke or heart attack.
In addition, "mortality probabilities" are a lot harder to psychologically absorb when you are 50 than when you are 20.
By the time you are 50, you've probably had more than a few of your friends and family in a coffin from a "low probability event".
Emotions are difficult. At some point, I will get diagnosed with a positive prostate screen (practically all men who live long enough do) that will almost certainly be "We should keep an eye on it." I hope I can weather it with the aplomb of my engineering background. We shall see.
Upvoted, but I remain skeptical. "You might have cancer but the probability is low and we should just watch it" is true for everyone who hasn't been screened, just the "might" is somewhat higher for those with a bad test result. Benign-looking moles on your arm could be cancer. Things you can't see or feel in your organs could be cancer. Also, plenty of other things could kill you practically without warning—heart attack, stroke, car crash, and the chance of the first two (and cancer) goes up over time. It seems to me that people have to come to terms with these things eventually, and "you have a low probability of having this type of cancer" just doesn't seem that different to me (perhaps it's best phrased as "you have an x% chance of being killed by this cancer"). If you've seen several people respond badly despite understanding probability, well, it's hard to argue with that, but I can't help wondering if it's fixable through slightly better communication or education.
Doing some reading about prostate cancer specifically, it seems like it actually is of the form "there is no appropriate action to take, regardless of the outcome of the screen"—or at least it's close enough that some experts think so. "In 2012 the United States Preventive Services Task Force (USPSTF) recommended against prostate cancer screening using PSA.[39] As of 2018 a draft for new recommendations suggests that screening be individualized for those between the ages of 55 to 69. It notes a small potential decrease in the risk of dying from prostate cancer, but harm from overtreatment." And: "A study in Europe resulted in only a small decline in death rates and concluded that 48 men would need to be treated to save one life. But of the 47 men who were treated, most would be unable to ever again function sexually and would require more frequent trips to the bathroom." https://en.wikipedia.org/wiki/Prostate_cancer_screening#Guid...
> "Didn't I just take a cancer screen? What was the result?"
“The result was ‘probably not,’ but the other option is ‘almost certainly not’ so we’d like to do more tests to make sure.”
(If the patient presses for more information, you don’t need to pull out the textbook yet; a simple analogy to explain why a cancer screen needs more tests might be by analogy to a rock screen, which sometimes catches clumps of dirt that need to be prodded a bit more to determine that they’re not actually stone.)
The important bit is to present it as routine and not yet a cause for concern, to avoid causing excessive alarm.
This test is a screening test and thus can never be used to diagnose someone. Only a diagnostic test can be used to diagnose someone.
Screening tests like these will represent a paradigm shift in cancer surveillance and as such will require those administering the test, most likely a pcp, to properly educate each patient about what the test results mean and what steps should be taken next.
The false positive rate doesn't depend on the number of true positives in the population. However, as you rightly point out the metric that actually matters for these kinds of screening is the precision/positive predictive value, and that one does.
The issue is that there's no follow-up test, at the moment. The cancer is otherwise undetectable.
...and the test does not determine location, so it's impossible to treat even with a positive test.
So from a clinical standpoint it's nearly useless, except to schedule patients for more frequent traditional testing. ...the counter point is that the stress caused to a patient is probably not trivial, even from a physical health perspective.
...but from a scientific standpoint, it's awesome and well worth further study.
Being able to catch a cancer early on because of increased surveillance is hugely valuable. Part of why cancer is so dangerous is because it often grows/spreads for years before you get any symptoms. If you're in the unlucky cohort of <50 years old and have an asymptomatic cancer, it might go decades before anyone thinks to check.
And mind, increased surveillance typically means once a year tests or scans, so it's not so overly onerous. It's possible even that generally being in frequent contact with doctors and more seriously minding your health could have net benefits in preventing other maladies as well.
You actually could potentially determine location by examining mutation patterns, both by looking more closely at methylation and doing further sequencing of cell-free DNA.
I imagine that there will be a longitudinal trial for this test. Individuals will be tested at set intervals and the test results will be correlated with clinical outcomes. It is possible that enough blood samples have already been collected such these same correlates could be identified retrospectively.
As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.
P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general)
Lets say 1% true positive rate is a "useless test".
0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity)
X = 0.0005
Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014
[cite](https://www.cancer.org/cancer/stomach-cancer/about/key-stati...)
Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that.
This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.
The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence:
The introduction of the test itself may change its effectiveness (reducing it). Cancer is an evolving condition that your immune system fights. Perhaps it's not very useful to characterize the test before it's applied since the test itself may change its efficacy. If you start exercising and dieting in response to the potential diagnosis it'll definitely help.
If we had a test that detected everyone and they believed it, we'd improve cancer rates.
But, the even more important metric to measure is mortality. Detecting a cancer that is 100% fatal 1 day earlier than traditional tests obviously wouldn't be that impressive. Even if the PPV is very high.
It's also very possible that a cancer that is detected accurately 6 months earlier, but that detection still doesn't improve our ability to treat the cancer. In these cases, it might even look like mortality is improved if you measure time from detection to death ("early detection extended survival by 6 months"). But, actually, it didn't change much.
So, that's all to say that the "Experimental" in the title should temper expectations considerably, because that has real meaning.
Eventually, this is all about cost. cfDNA assays are targetted to in the end cost less than 100 bucks per sample which for the detection of a single cancer puts you in the range of $12000 for one tumor diagnosis. Ideally, you can now decrease the therapy cost of that one diagnosis while also boosting survival chances. And $12000 is not much in tumor therapy.
The key however here is the advanced prognosis that may allow totally different therapeutic approaches, extend the understanding risk factors etc. and the fact that you do not need to have a specialist on side, taking a blood sample can in principle be done at home and the rest is automatic.
Edit: Oh, and 5% false positives is not that bad actually, I do not have numbers is my head but would expect other essays like breast cancer screening to be also in that range.
The problem is with breast cancer screening the next step probably isn't a full body CT. If you detect cancer in the blood, next step would be actually finding it. A full body CT would actually increase your cancer risk and add the chances for another false positive which might involve more invasive diagnosis. The real risk/benefit calculations quickly become tricky.
All that said, I'd rather risk dying while propofol'ed, than from say colon cancer. I mean all this is about how we die, not if we have to die.
Coming from the field of cognitive neuroscience, where MRI scans are commonplace (because you can get an estimate of brain activity by doing a continuous, "functional" MRI scan), CT feels like ancient technology to me. There are highly specific case where it has benefits, so it has a place. But if I went into the hospital and they sent me in for CT rather than MRI to get an anatomical scan, they better have a damn good reason for it.
Well, you better be willing to wait a couple days for a non-emergent MRI.
Because MRIs take longer, they are a more scarce resource. If you are in the hospital and need a scan, presumably its because the doctors are trying to diagnose something somewhat urgently. If not, they should discharge you and let you get the scan scheduled as an outpatient. So, CT it is for almost all common conditions in the hospital as first line. (even stroke usually gets CT head first to rule out a bleed, where the sensitivity is still pretty good)
CT and MRI are simply different imaging modalities that provide different information, usually used to answer different questions. There are tradeoffs for each.
By the way, they were both invented around the same time (CT is only older by like ~5 years).
CT is simply more practical for full body scans, since an MRT would take forever. Imagine laying completely still for 2 hours... CT does it in 5 minutes.
They also differ in diagnostic value for different types of use cases. But I am lacking any expertise there.
Either way, CT plus constrast agent is the standard in cancer diagnostics/staging.
Excuse my ignorance, but I've always been told that CTs are significantly cheaper and faster to administer. According to the famous XKCD radiation graph, a full CT head scan is about 50% of the normal natural yearly background radiation you'll get. Not something to cheer over, but nothing to lose sleep over either. If a CT scan would suffice, why tie up the MRI machine?
The problem is when you are screening lots of people, most of whom don't actually have cancer. The individual risk is low, but the population risk is high. And when we talk about cancer screening we have to think at the population level.
From what I know, the risk is not high enough to leave a strong signal. The problem is the overall high cancer rate for humans. As far as I know, we still don't have a model for predicting cancer rates after low exposure to ionizing radiation.
OK, but you'd be delaying scans by having to wait for the MRI, which takes significantly longer to setup and administer as well as being much more expensive (which would further reduce access).
Also (and this is according to the XKCD chart) you'd need 50 CT head scans before you'd hit the clear statistical cancer risk.
But it's not like the whole population gets a CT scan every year, or even during their entire life. If you need one anyways, there's probably a more pressing issue that 1/50th the clearly statistical risk level.
The dosage is in a shorter amount of time (within seconds instead of months). I don’t think the radiation averaged over half a year is comparable to the head CT.
My understanding is that the dosage within those few seconds is the equivalent to absorbing half a year's worth of radiation. That's explicitly what Sieverts are designed to measure: https://en.wikipedia.org/wiki/Sievert
It's disputed, if the linear, no threshold model is adequate. The problem is, 1:5 people die of cancer anyway and the people getting fullbody CTs tend to be either old and die rather soon, or are people who had cancer at some point to begin with. Really hard to find a signal there. The best data is still from the nuclear accidents/bombings.
Cells can repair (maybe faulty) a certain amount of damage, but may suicide when too much is broken (double-strand breaks). The amount and type of ROS generated by ionizing radiation also depends on your antioxidant state and how well tissue is saturated with oxygen (more ROS if you exercised before exposure). Generated ROS are a significant factor in cell damage, it's not just direct DNA hits. Some ROS can last for weeks and travel across cells to fuck things up.
I think we can confidently say, lowish radiation exposure is: not great, not terrible ;)
5% false positives on people who have some symptoms of cancer isn’t bad. But 5% of a ‘high risk group’ could be a lot - particularly if the cancer is too early to confirm detection any other way yet and possibly not treatable for several years.
But in this example - stomach cancer - you could just skip the test and do a routine endoscopy and get some tissue samples. You’d have a much higher confidence in the result.
Reminds me a bit of the PSA test for prostate cancer. More of an early warning, but with the weak predictability, doctors are moving away from routine PSA tests.
If option A is regular blood tests with an endoscopy to confirm should the blood test come back positive and option B is getting endoscopies on a regular basis, option A is a no brainer. While it might not be super invasive, I'd definitely like to keep the number of times a tube has to be shoved down my throat to a minimum.
No, the false positive rate is irrelevant. Let's say you want to check for cancer annually and the blood test has a 50% false positive rate:
If you only get endoscopies after positive results, you get an endoscopy every other year. If you skip the test and go straight to the endoscopy, you get one every year. Thus you get half as many tubes shoved down your throat if you get the blood test.
For a more reasonable false positive rate like 5%, you would have a 50/50 chance of getting a false positive any given decade. I'd much rather get 4 endoscopies than 40.
Indeed, for any false positive rate less than 100% (which isn't really a test to begin with) still on average decreases the number of endoscopies you need.
Really the metric that's important is the false negatives: if that's too high then I can't safely forego the endoscopy just because this year's blood test was negative. Even then though, if the blood test has a long lead time then maybe it's safe to wait and see how next year's blood test goes.
Even if the first option is "get more regular checkups over the next year+, to characterize the growth rate."
You don't have to nuke an organ from orbit, if you have historical data that indicates a tumor isn't aggressive. Or you would, if you've been watching it and it suddenly takes off.
The problem is that you don’t want to treat every cancer like it’s potentially life-threatening (many aren’t) because you can cause other problems with detection/treatment. I’d want any early detection to be pretty damn solid if we’re going to start talking monthly checkups for something that could resolve on its own.
Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week.
Another potential benefit of this sort of early detection would be the ability to more effectively test interventions that might have little effect on more advanced cancers.
That could help direct research into the development of cancers at its earliest stages, and potentially pave the way for more effective prevention.
If you need 12k$ for a single tumor diagnosis, then we should consider MRI imaging. Those cost significantly less than that. And as of right now, the positive effects of chemotherapy on such early cancers would be debatable. Still a useful development though.
Worth noting an MRT is of low value for prevention. Small lesions would be invisible and you could very well progress from nothing visible to seriously sick in 6 month. We really need some marker for before we get unhindered growth.
Even if we had 100% precise AI assisted image analysis, once a year fullbody MRI... the MRI resolution would still not catch the "before things escalate quickly" pre stage of cancer. With imagery catching _most cancers early is inherently a thing of mostly luck.
You misunderstand. 12k$ (or some number in that ballpark) would be per postive test for testing everyone. Your 12k$ for MRI would check a single person. That's something totally different.
The point is that treating a cancer patient easily costs tens or of hundreds of thousands plus dire consequences for the patient themselves. So there is a lot of room for
improvement if the diagnosis is made that much earlier.
Blood tests for testing cancer markers during therapy are already standard since a long time. A family member of mine got that testing regularly about 10 years ago but the predictive value was not good. Still, if the accuracy improves it's a good development and could be revolutionary once high enough since some types of cancer are very likely to be curable if detected early enough. But yes, a false positive result can make things worse, especially for people that are in a risk group.
My understanding is that there is some dependency on the type of cancer, but mostly it depends on your immune system response. Doctors are pretty clear that just having a cold can affect the numbers.
Personally I'd like to know I'm in really early stages.
I would take positive steps waiting for a retest, such as cutting out refined carbs, saunas or other heat therapies, and eliminating stress and other harmful situations.
false positive? probably still a better lifestyle will have it's own benefits.
That would only help if the false positive occur completely at random. If the error is systematic (for example because the test is detecting something else in the blood) you'd just keep getting false positives.
presumably these other things that would cause repeated false positives would be identified over time and could be ruled out with other tests - ie you either have cancer or a serious iron deficiency, let's check if you have an iron deficiency.
It probably depends on what is causing the false positive.
If it is just some variance in the test, then multiple testings could help (And the test could probably be improved to reduce false positives pretty easily)
But if there is something intrinsic to those tested (say, some hormone level or other) Repeat tests will give repeat false positives. AND it may be harder to eliminate false positives by refining the test
The false positive rate reported is ~5% in a sample with an equal number of people that did and did not develop cancer, while the ratio in reality is 1/122 (i.e., 1000 people developed cancer, 122000 did not).
Could still be a useful development, but it does make me wonder about the cost/benefits at this stage.