Recently I wrote a post discussing overdiagnosis in the workup for pulmonary embolism (PE). In it we discussed a recently published paper, the RESPECT-ED trial. Shortly after posting I was contacted by the lead author, Dr. David Mountain, with a number of comments and concerns on how the trial was represented. I have included it here as it is a wonderful assessment.
Dear Rory
Many thanks for reading/reviewing our paper RESPECT-ED (1). It is always good to get any feedback.
With regards to your commentary I will go thru in point form some of the issues raised- these might form the basis of longer conversations in the future
1. Overdiagnosis with any test with < 100% specificity is inevitable. The issue is when does the level of over-diagnosis become harmful, outweigh benefits (discuss in < 500000 words!)
2. The aim (goal) of our study was not to quantify the over-diagnosis rate in current PE diagnosis with CTPA AND we never stated this as a goal. Our study did not have methods allowing this. We instead looked to see if there was significant variation in yield AND if there were significant numbers of departments with low rates of yield- these are at best potential indicators of overdiagnosis.
3. We did hypothesise (as suggested many times in the PE overdiagnosis literature ) that ED with low yield and /or high use, using newer CTPA may have higher proportions of SSPE/ small PE. We looked for correlations between CTPA use, low yield and small PE rates but found none for any of these potential associations at our rates of use/yield (an important caveat). We did not state anywhere that small PE were clinically insignificant in either introduction or aims. No definition of clinical insignificance was stated in our methodology as we did not follow patients for clinical outcome. However we do discuss if maintained large PE proportions, even at sites diagnosing more PE with > use, suggests they may be considered clinically relevant.
4. The conclusion does not assume size is the “only” determination of clinical relevance , that is a reviewer inference. Clearly co-morbidity, genetics, cardio-respiratory reserve, RFV function, Inflammatory responses etc are important. Size is not a clear indicator of overall mortality although it is associated with more complicated hospital course (2)
5. Dan Exter’s study on SSPE vs larger PE vs no PE, concludes that we can’t ignore SSPE because they worse clinical consequences than no PE patients.(3) However our discussion and findings never suggest ignoring SSPE or that they are always clinically irrelevant. We instead looked to see if SSPE rates, where false +ves are most frequently seen (up to 50%), increased with high resolution CTPA, particulalrly with higher use /lower yield sites. None of these potential correlations were found at our levels of yield and usage.
6. We draw your attention to the carefully worded discussion on the issues of overuse and low yield. (1) We made it absolutely clear that our findings would probably not be applicable to environments where CTPA use was 2-3 times greater and yield 2-3 x lower, such as many US EDs. However within the range of Australsian practice seen we found little evidence for previously suggested markers of over-diagnosis e.g. > proportions of small PE at sites with high use and/or low yield.
7. As the reviewer rightly points out direct measures of overdiagnosis are hard to find- although not as impossible measuring a black hole. True overdiagnosis (false positives) were reported in two recent papers with high SSPE rates (20-25% of PE) where 40-50 % of SSPE were definite/probable FP on multiple C-thoracic radiologist review. (4,5) We agree definitive measures of over-diagnosis are elusive or defective- particularly those relying on flawed population level post-mortem data to infer over-diagnosis. (6) We note that this data applies to an environment of use/ yield that we think we can safely suggest is not the same as Australasia’s. An plausible alternative hypothesis for reduced case fatality rates is that early / massive PE deaths that are easy to diagnose, and hard to treat, so will be relatively constantly reported BUT that as more PE are diagnosed and treated with effective therapies that lower mortality rates this would obviously reduce overall case mortality.
In summary although we are pleased to have our paper reviewed, and to spark important discussions we think it is important that our actual aims, methods and discussions are represented accurately. We are very happy to discuss any of these issues further (perhaps one at a time!)
Cheers Dave Mountain MB BS FACEM, Univ of Western Australia–lead author RESPECT-ED study.
First I just wanted to thank David for his thoughtful response and hope he realizes that my comments were not directed towards the quality of the study itself, but rather how we quantify overdiagnosis. In fact, we could just as easily discussed these same concepts in relation to the now infamous PESIT trial.
While the RESPECT-ED cohort failed to find an increase in the proportion of subsegmental PEs (SSPE) diagnosed in sites with high use/low yield, they did observe that the more CTPAs that were performed, the greater the number of PEs that were identified(1). Does this mean that low use centers are missing clinically important diagnoses? Or are the high use/low yield centers diagnosing clinically unimportant PEs? Maybe the diagnostic sweet spot is somewhere in the middle? If so where? Or maybe there is considerable variability in the prevalence of disease between the sites examined? Or just maybe, the more we go looking for PE, the more often we will find it, and the clinical importance of these additional diagnoses is uncertain.
Before going any further I think it is important we distinguish overdiagnosis from misdiagnosis. We often confuse overdiagnosis and false positive findings, which are not necessarily the same thing(7). False positives account for situations in which a test like a CTPA is positive for a clot when in reality one does not exist. Overdiagnosis is subtly different. Overdiagnosis occurs when the test, CTPA in this case, finds a true clot, but that clot is of no clinical consequence to the patient. The two studies cited by David in his letter are wonderful examples of the tendencies of subsegmental emboli to be false positives, but they do not address overdiagnosis. The first of these studies, Hutchinson et al, was published in the American Journal of Roentgenology in 2015(4). These authors conducted a retrospective analysis of all CTPAs performed in a tertiary hospital over a 12-month period. Studies that were originally found as positive were reread by 3 chest radiologists. Overall 937 CTPAs were identified, 174 (18.6%) were classified as having a PE on the original radiologist’s read. 25.9% of these cases were reread as negative when the chest radiologist reviewed the images. This occurred most frequently in segmental (26.8%) and subsegmental (59.4%) clots. Miller et al performed a similar study, finding similar results. 15% of the subsegmental defects were reread as negative on the second evaluation(5). And so while both these studies are fine examples of the risk of false positive findings associated with small PEs, they do not examine the prevalence of overdiagnosis.
The reason overdiagnosis is so hard to identify is because unlike false positives it rarely has a gold standard with which to identify its presence. As soon as a blood clot is identified, it is considered a true finding and is treated like any other pulmonary embolism. This is doubly true for larger clots, because of the bias that clot size is inherently tied to clinical importance. On an individual level it is impossible to know if any one clot, SSPE or otherwise, would have had clinically important consequences. Instead we have to examine surrogate data.
First how do we know that overdiagnosis exists?
Since the introduction of CTPA into clinical practice in 1998, the number of PEs diagnosed has risen by 80%. This number was essentially stable the five years prior to the introduction of CTPA (8). In their analysis of this data, Weiner et al offer a number of possible explanations(6).
The first is an increase in risk factors results in more pulmonary emboli, with no change in disease severity. If this was the case then the incidence and mortality should increase with no change in the case fatality rate.
The second scenario is the one David suggested, that the more sensitive test detects more pulmonary emboli, and new cases benefit from treatment leading to improved outcomes and fewer deaths. If this were the case then we would see an increase in incidence of the disease but a decrease in both the mortality and the case fatality rate. We have not.
The final suggestion is far more consistent with the observed data. The more sensitive test combined with our more liberal testing strategies detects more pulmonary emboli, but these new cases do not necessarily benefit from treatment (overdiagnosis). In such cases we would observe an increase in incidence and a decrease in case fatality, but no change in overall mortality. This is exactly what Soylemez et al observed.
Probably the best direct data we have for the existence overdiagnosis is an RCT published in JAMA in 2007. In this trial Anderson et al randomized 1417 patients at high risk for pulmonary embolism to undergo a diagnostic workup with either a CTPA or a ventilation perfusion scan. While the author found no difference in mortality or rate of subsequent VTE, the authors did observe that patients in the CTPA arm had a significantly higher number of PEs found on initial scanning(12).
What evidence is there to suggest that overdiagnosis is limited to SSPEs?
What den Exter’s study demonstrated was that it is not the size of the clot which determines patient outcomes, but rather the patient level-factors(3). Sick, old patients, with multiple comorbidities get true clinically important PEs. In fact even when they were ruled out on the initial visit, their 3-month mortality was fairly similar. Conversely, young healthy patients with no reason to get PEs do well, independent of the size of the emboli. Essentially it is the patients and not the size of the clot that determines outcomes.
A number of individual centers have examined the increased trends of CTPA utilization over time and found a concordant increase in the number of PEs diagnosed without any change in overall mortality. And while they did find an increase in the number of SSPEs, they also found an increase in the number of larger PEs diagnosed, suggesting that overdiagnosis is not limited to subsegmental clots. Aurer et al examined a prospectively collected a database of postoperative cancer patients and found the rate of CTPA increased from 6.6 CTPAs per 1,000 patients in 2000 to 44.7 per 1,000 patients in 2005(9). This coincided with an increase in the number of PEs diagnosed, which rose from 2.6% in 2000 to 9.3% in 2005. The percentage of patients diagnosed with a subsegmental PE rose from 0.15% in 2000 to 3.0% in 20005. Likewise the number of patients found to have a segmental embolism in 2000 was 1.76% compared to 5.32% in 2005.
Additionally, two prospective trials which examined patients who underwent screening for PE, with no clinical suspicions, found that the clot burden discovered in this asymptomatic cohort was not limited to the subsegmental branches of the pulmonary vasculature (36.3% and 78.6% of the PEs diagnosed in each respective were located in the segmental or central vessels)(10,11).
Overdiagnosis is a subtle and elusive scoundrel. Its influences are far reaching and are not limited to subsegmental clots. Anatomical definition of clot burden alone, cannot tell us which PEs are clinically relevant and which are simply stumbled upon due to an overeager diagnostic pathway.
Sources Cited:
1. Mountain D, Keijzers G, Chu K, et al. RESPECT-ED: Rates of Pulmonary Emboli (PE) and Sub-Segmental PE with Modern Computed Tomographic Pulmonary Angiograms in Emergency Departments: A Multi-Center Observational Study Finds Significant Yield Variation, Uncorrelated with Use or Small PE Rates. PLoS ONE. 2016;11(12):e0166483.
2. http://www.sciencedirect.com/science/article/pii/S0002934315000972 The American Journal of MedicineVolume 128, Issue 7, July 2015, Pages 747–759.e2
3. Den exter PL, Van es J, Klok FA, et al. Risk profile and clinical outcome of symptomatic subsegmental acute pulmonary embolism. Blood. 2013;122(7):1144-9.
4. Hutchinson BD, Navin P, Marom EM, Truong MT, Bruzzi JF. Overdiagnosis of Pulmonary Embolism by Pulmonary CT Angiography. AJR Am J Roentgenol. 2015 Aug;205(2):271–7. doi: 10.2214/AJR.14.13938. pmid:26204274
5. Miller WT, Marinari LA, Barbosa E, Litt HI, Schmitt JE, Mahne A, et al. Small pulmonary artery defects are not reliable indicators of pulmonary embolism. Ann Am Thorac Soc. 2015 Jul;12(7):1022–9. doi: 10.1513/AnnalsATS.201502-105OC. pmid:25961445
6. Weiner RS et al. Acute pulmonary embolism: comment on “time trends in pulmonary embolism in the United States”. Arch Intern Med. 2011;171(9):837-9.
7. Walker MJ, Rogers W. Defining disease in the context of overdiagnosis. Med Health Care Philos. 2016;
8. Wiener RS, Schwartz LM, Woloshin S. When a test is too good: how CT pulmonary angiograms find pulmonary emboli that do not need to be found. BMJ. 2013;347:f3368.
9. Auer RC, Schulman AR, Tuorto S, Gonen M, Gonsalves J, Schwartz L, et al. Use of helical CT is associated with an increased incidence of postoperative pulmonary emboli in cancer patients with no change in the number of fatal pulmonary emboli. J Am Coll Surg2009;208:871-8; discussion 878-80
10. Ritchie G, McGurk S, McCreath C, Graham C, Murchison JT. Prospective evaluation of unsuspected pulmonary embolism on contrast enhanced multidetector CT (MDCT) scanning. Thorax2007;62:536-40.
11. Schultz DJ, Brasel KJ, Washington L, Goodman LR, Quickel RR, Lipchik RJ, et al. Incidence of asymptomatic pulmonary embolism in moderately to severely injured trauma patients. J Trauma2004;56:727-3
12. Anderson DR, Kahn SR, Rodger MA, Kovacs MJ, Morris T, Hirsch A, et al. Computed tomographic pulmonary angiography vs ventilation-perfusion lung scanning in patients with suspected pulmonary embolism: a randomized controlled trial. JAMA2007;298:2743-53.
- EM Nerd-The Case of the Partial Cohort - May 24, 2020
- EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
- EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019