Published in the journal JAMA Internal Medicine. Here is a link to the article.
Regenstrief Institute authors: Tom Imperiale, M.D.
Abstract
Both the fecal immunochemical test (FIT) and multitarget stool DNA test (mt-sDNA) are stool-based screening tests for colorectal cancer (CRC) that are reported qualitatively as positive or negative, even though both tests are quantitative, producing a precise value on a continuous scale. The FIT quantifies the globin portion of human hemoglobin in either per milliliter of buffer or per gram of stool, while the mt-sDNA test generates a score from a multivariable algorithm that measures human hemoglobin as well as specific abnormalities in human DNA. For both tests, the threshold (ie, the cut point) determines how the test performs, ie, its sensitivity and specificity. Lowering the threshold increases sensitivity for CRC, advanced precancerous lesions (APLs), or both and would likely decrease specificity, resulting in a higher false-positive rate and more (unnecessary) colonoscopies.
The recently published BLUE-C study1 was a clinical trial comparing a next-generation mt-sDNA (ng-mt-sDNA) test with a commercial FIT (OC-AUTO FIT; Polymedco, which has a threshold of 100 ng/mL, roughly equivalent to 20 μg/g) in nearly 21 000 average-risk persons who were scheduled for screening colonoscopy. The ng-mt-sDNA test had superior sensitivity to the FIT for both CRC (94% vs 67%, respectively) and APLs (43% vs 23%, respectively), while FIT had superior specificity (95% for FIT vs 90.6% for ng-mt-sDNA). Is it possible that lowering the FIT threshold could increase sensitivity to the point of equivalent performance with the ng-mt-sDNA test?
In JAMA Internal Medicine, Seum and colleagues2 report results of a comparative analysis of BLUE-C study participants and the authors’ BLITZ cohort, which consists of nearly 7200 patients screened with colonoscopy who completed testing with a commercial FIT (FOB Gold; Sentinel Diagnostics) prior to colonoscopy. The authors tailored the BLITZ cohort to make it comparable to the BLUE-C participants both demographically and in the prevalence of the most advanced finding on colonoscopy. While the sensitivities of FOB-Gold for CRC and APLs at the manufacturer’s recommended threshold of 17 μg/g were lower than those of ng-mt-sDNA, when the threshold was lowered to 11.7 μg/g, which is the point of equivalent specificity with ng-mt-sDNA (90.6%), the CRC sensitivity of the FIT used by Seum et al2 remained the same, while that for APLs increased, narrowing the gap between the two tests from 11% to 5%. Further lowering the threshold to 10 μg/g produced similar APL sensitivities between the FIT used by Seum et al2 and ng-mt-sDNA. Do these findings indicate that the ng-mt-sDNA test is just a complicated and expensive FIT?
Answering this question requires statistical and methodologic consideration. Statistically, we know that increasing a test’s sensitivity will decrease its specificity and vice versa. For the comparator FIT in the BLUE-C study,1 lowering its specificity to 90.6% (to equalize specificity to that of the ng-mt-sDNA test) requires lowering the threshold to 56 ng/mL (or 16 ug/g), increasing FIT sensitivity for CRC to 75.5% (95% CI, 65.8-83.6) and APL sensitivity to 31.8% (95% CI, 29.8-33.8), both of which are still inferior to those of ng-mt-sDNA. Further, applying the 13.4% positivity rate of ng-mt-sDNA to the comparator FIT (to equalize test positivity rates) requires reducing the FIT threshold to 47 ng/mL (or 12 μg/g), increasing CRC sensitivity to 76.5% (95% CI, 66.9-84.5) and APL sensitivity to 33.6% (95% CI, 31.6-35.7), both of which remain inferior to the ng-mt-sDNA test. Thus, from a statistical perspective, the comparison between the ng-mt-sDNA and the comparator FIT indicates that the ng-mt-sDNA provides better discrimination than the comparator FIT and is not just a complicated and expensive FIT. Why, then, are the findings of Seum and colleagues2 so different from the same type of analyses from BLUE-C? Here is where methodologic consideration is required.
While the reason(s) for the difference is not totally clear, there are at least 2 possibilities. The more important one is that the comparison of ng-mt-sDNA and commercial FIT by Seum and colleagues2 was conducted in 2 different cohorts, which, despite their comparability demographically and on the most advanced colorectal findings, may differ in other characteristics that might be related to test performance such as lesion size, location, concurrent findings, and colonoscopy quality metrics. A second reason may be that, despite having similar positivity thresholds (of 17 μg/g and 20 μg/g), the FIT used by Seum et al2 and the FIT used in the BLUE-C study1 have different applied performance characteristics, which are determined not just by threshold but also by factors such as time from specimen collection to processing, ambient temperature, and whether the specimen was tested fresh or after freezing and thawing.3
While statistical comparison of screening and diagnostic tests is necessary for helping us understand and compare test performance, it is not sufficient, as good study design and analysis are also required. Nonetheless, the findings of Seum and colleagues2 are provocative and highlight the need for rigorous and valid comparisons of noninvasive CRC screening tests, which require that the tests are compared not just in the same cohort, but on the same specimens (stool or blood) from that cohort. In a recently published study, Levy and colleagues4 used this rigorous study design to compare the performance of 5 different FITs in detecting advanced colorectal neoplasia (the combination of CRC and advanced precancerous polyps), enrolling 3761 participants who completed all 5 FITs prior to preparing for their screening or surveillance colonoscopy. The study findings support the conclusion that FITs have varying test performance and are not interchangeable. Given the variation in test performance across noninvasive tests, whether due to intrinsic (ie, test threshold) or extrinsic (ie, study population, time from collection to testing, ambient temperature, and other) factors, more direct studies like that of Levy and colleagues4 are needed to evaluate comparative test performance of noninvasive screening tests for CRC.
Authors
Thomas F. Imperiale, M.D.1,2,3,4
Author Affiliations
1Division of Gastroenterology and Hepatology, Department of Medicine, Indiana University School of Medicine, Indianapolis
2Center for Health Information Communication, Health Services Research and Development, the Richard L. Roudebush VA Medical Center, Indianapolis, Indiana
3Regenstrief Institute, Inc, Indianapolis, Indiana
4Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, Indiana
