Since Hector Pope published his now infamous study in the April 2000 NEJM, Emergency Physicians ability to clinically differentiate ACS has been called into question(10). Despite the fact the study's findings that ED physicians missed an almost flawless 0.18% of chest pain patients who went on to be diagnosed with a MI, we have been told we are unable to trust our clinical judgement. We have been taught women present atypically, the elderly present atypically, diabetics present atypically. We have been taught that reproducible chest pain does not rule out ACS, that indigestion does not rule out ACS, and that the lack of chest pain all together does not rule out ACS. We are told that we can never fully exclude unstable angina, a clinically diagnosis that we have been made to believe we cannot clinically preclude. We are being asked to do the philosophical impossible, prove a negative. Though there is some truth to these warnings, we have taken these admonishments to the extreme. Chest pain has become a black box of sorts with which no amount of risk stratification and clinical judgement will suffice. As these teachings are passed from one generation of residents to the next, our comfort with clinical decision-making in ACS has become progressively more strained. So like any other unknown we attempt to quantify the problem. Identify the risk. Objectify our findings. We create a decision instrument or in this case a diagnostic pathway. The trials discussed below are a reaction to our fear, our uncertainty, and our discomfort with the unknown. This diagnostic conduit is a response to being constantly told our intrinsic abilities are just not good enough. For better or worse we have brought this on ourselves…
In October 2013 JAMA Internal Medicine published an article by Than et al validating the concept that if you take a low-risk chest pain cohort that is so low risk no one has the disease which concerns us, than it doesn’t matter which tests you perform, everyone will do fine(1). This trial entitled “A 2-Hour Diagnostic Protocol for Possible Cardiac Chest Pain in the Emergency Room” was an attempt to validate the findings from a group of consanguineous cohorts called the ADAPT, APACE and ASPECT trials. Their common goal to identify a group of patients appropriately low risk that they could be safely discharged from the ED using a 2-hour protocol, which they called Accelerated Diagnostic Protocols (or ADPs).These 3 cohorts have been published in various forms over the last few years (2,3,4,5). Most recently their findings were combined into what was called a “validation set” of high sensitivity troponin I in a 2-hour rule out strategy (6).
Using EKG, change in high sensitivity troponin (delta trop) and the TIMI risk score, the authors were able to identify a small group of patients who were safe to discharge without further testing. This should not be mistaken for success. The authors knew that even in this low risk cohort with a15.3% overall event rate most of which (13.8%) were NSTEMIs, that none of the multitudes of cardiac enzymes they tested were sensitive enough to effectively rule out ACS. In the original publication of the APACE cohort from 2009 NEJM(2), the authors examined the diagnostic accuracy of these various enzymes. The best out of these markers was the Roche-High Sensitivity Troponin T with a sensitivity of 95% and a specificity of 80% at 6 hours. At the 2-hour interval, the time period of the proposed protocol, the sensitivity dropped to 89%(3). Clearly when used on a low risk population, even the ultra sensitive troponins are not sensitive enough to effectively rule out ACS. To compensate for this, the authors attempted to identify a subset of their cohort whose pretest probability was low enough for the 2-hour troponin to have adequate negative predictive value to rule out a bad outcome over the next 30 days. To identify this ultra low risk subset the authors used a TIMI risk score of zero.
The TIMI risk score is a seven part decision instrument that allots one point for each variable. The higher the score the greater the chance of a poor 30 day outcome. The TIMI score was initially derived as a prognostic measure in NSTEMI and unstable angina patients. It was developed during prior trials comparing enoxiparin to heparin. The authors of these trials wanted a fairly high risk group of ACS patients so as to demonstrate the greatest degree of benefit for their fancy new drug. Also known as high acuity bias. Using various cohorts from the TIMI 11B and the ESSENCE trials the authors derived and then validated their rule. In these derivation and internal validation cohorts there was a consistent progressive increase in the risk for mortality or MI over the next 14 days with each incremental rise in the TIMI risk score (7). What is important to note that these were not the undifferentiated chest pain patients we see in the emergency department but rather patients diagnosed with NSTEMI or unstable angina. It was not until Chase et al published an article in The Annals of Emergency Medicine in 2005 that the TIMI risk score was evaluated prospectively on undifferentiated chest pain patients(8).
The Chase et al cohort consisted of 1,458 patients in which a total of 136 adverse events occurred. Among the subgroup with a TIMI risk score of 0, there were 8 adverse events over the next 30 days(5 AMI, 2 revascularizations and 1 death). Excluding the 2 revascularizations (very subjective outcome) the event rate in this ultra low risk group was 1.2% and the mortality rate was 0.2%. This is the baseline risk level that we are dealing with when we approach this very low risk subset of chest pain patients.
Now some would argue a TIMI score of 0 significantly risk stratifies patients to a level where no further testing is required. Pointing out that the rate of false positives in this cohort and all the resources and tests performed can be far more harmful than the questionable NSTEMIs and very rare death that would be missed if this quixotic search was not undertaken. In the combined cohort of the ADAPT and APACE trials (6) the ADP negative cohort (TIMI=0, -EKG, and – hs-TnI) had 100% sensitivity but its specificity was dismally low at 23.1%. A TIMI score of zero alone has a sensitivity of 98.4%. Meaning you have to test 63 patients with high sensitivity troponin to identify one additional NSTEMI. Along with this true positive you would admit 42 false positives to receive all the harms of anticoagulation, dual antiplatelet therapy and the risk involved with further provocative testing and invasive interventions. 3/4 of your chest pain population would fail this pathway and require further testing. The authors proposed increasing the low risk group to include TIMI risk scores of 0 and 1. Using this criteria the ADP now boasted a specificity of 48.7%, with very little change to its specificity (99.2%). Overall this modified pathway missed 3 adverse events, as explained by the authors in the supplementary appendix, were of questionable clinical significance(9).
Despite their poor specificity, the results from the ADAPT, APACE, and ASPECT cohorts and their combined data set are promising. These cohorts were utilized in multiple publications, with multiple cardiac enzymes examined and multiple diagnostic pathways explored in an attempt to identify the most statistically efficient protocol. As we know manipulating data in this manner can lead to misguided findings that can very easily be due to chance alone. Knowing that these findings were only hypothesis building the authors set out to test the hypothesis ADAPT and its sibling cohorts developed.
It was this hypothesis validating study that was published in JAMA Internal Medicine on October 7th (1). Using the ADP originally developed in ADAPT's original trial design, to be low risk the authors required a TIMI risk score of 0, a non-ischemic EKG, and a negative delta troponin I at 2-hours. Unlike their original cohorts this was an RCT that randomized patients to a standard workup vs the 2-hour rule out protocol. Their primary outcome measure was successful discharge within 6-hours without subsequent 30 day major adverse cardiac event (MACE). MACEs were defined as death, cardiac arrest, emergency revascularization procedure, cardiogenic shock, ventricular arrhythmia, and AMI. I would argue that most of these adverse events seem like reasonable endpoints with the exception of emergent revascularization (as we know very subjective and variable) and to some degree AMI (many are troponin leeks of little clinical relevance).
Of the 542 patients which made up the entire cohort there was a 14.9% rate of MACE, almost entirely occurring during the initial hospital visit. Only one patient experienced an adverse event over the 30 day follow-up period. He was a 65-year-old male who following his negative ED ADP, had a negative stress test and 6 days later returned with a ST-elevation MI. There was obviously a statistical difference in the amount of patients discharged from the hospital within 6-hours. More importantly there was no difference in the rate of adverse events between the two groups. 87 patients were classified as low risk by the ADAPT groups criteria. The same number that was classified by a TIMI score of 0. This means that neither the EKG or the troponin played any role in the management of these patients. The rule would have performed just as well if all patients with a TIMI score of 0 were discharged home without any further testing. This is not a demonstration of how well this rule performs in the clinical arena, but rather an example of how any rule can look excellent when tested on a super low risk population. Say for example I were to test a new form of birth control and measure its success by the rate of pregnancy in my test population. It would not be sincere if I proclaimed success after testing this new treatment in an all male cohort. This is similar to testing a ACS rule out criteria in a population where so few actual cardiac events occurred. A population where the majority of events were NSTEMIs of questionable clinical relevance, 3 STEMIs, 1 ventricular arrhythmia, 3 AV blocks, 1 30 day MACE, and no deaths.
The most important question is how does the emergency physician perform using his or her unstructured judgement? Are we really as bad as we have been told when diagnosing ACS? A study by Mitchell et al published in The Annals of Emergency Medicine in 2006 demonstrated just the opposite(11). In this prospective cohort physicians were asked to estimate the risk of ACS. When the physicians estimated the patient's risk of ACS to be under 2%, 0.7% of patients had a ACS related event in the next 45 days. When the physicians estimated the risk of ACS to be 2%, the 45 day event rate was 2% and when the physicians estimated the rate of ACS was over 10% the 45-day event rate was 17%. In October 2013 a similar article was published in The Annals of Emergency Medicine again asking if Emergency Physicians unstructured judgement was sufficient to rule out ACS. In this article they examined both clinical estimates of pretest probability for ACS and pulmonary embolism(12). Similar to the previous study the physicians were asked to estimate the risk of ACS. When physicians calculated the risk of ACS to be under 2.5%, the rate of 90 day ACS related events was 0.2%. When the physician felt risk was greater than 10%, the 90 day event rate was 7%. These two studies clearly demonstrate that the emergency physician is more than capable of ruling out ACS at a rate far below any realistic threshold.
Maybe the Than trial (1) is not a success for the ADP method of approaching chest pain, but it does suggest that there is an ultra low risk population in which no further work up is needed. There may be a diagnostic pathway that efficiently and accurately identifies this cohort, but I suspect like most complex diagnoses in emergency medicine, it will never perform as well as the unstructured judgement of the Emergency Physician. Instead of conducting more trials on various ADPs in pursuit of perfection, we should focus our attention on studies validating our clinical exam with real patient oriented outcomes and an acceptable miss rate. The future generations of Emergency Physicians should be taught that though ACS must be approached with care and respect, not every patient with chest pain needs to undergo a full barrage of cardiac tests. Philosopher Bertrand Russel wrote about his “Magic Teapot” which he claimed orbited the Sun somewhere between the Earth and Mars. He went on to say it would be nonsensical for him to expect others to believe in his Teapot simply because we cannot disprove its existence. The same stands for our chest pain patients. The presence of ACS should not be believed simply because we cannot disprove its existence in the emergency department. Chasing this “Teapot” through space and time has already lead us far astray and will surely lead to far more harm if we continue this futile search for perfection.
Sources Cited:
1. Than et al. A 2-Hour Diagnostic Protocol for Possible Cardiac Chest Pain in the Emergency Department: A Randomized Clinical Trial. JAMA Intern Med. 2013
2. Omland et al. A sensitive cardiac troponin T assay in stable coronary artery disease. N Engl J Med. 2009 Dec 24;361(26):2538-47
3. Reichlin et al. Utility of absolute and relative changes in cardiac troponin concentrations in the early diagnosis of acute myocardial infarction. Circulation. 2011 Jul 12;124(2):136-45.
4. Than et al. 2-Hour accelerated diagnostic protocol to assess patients with chest pain symptoms using contemporary troponins as the only biomarker: the ADAPT trial. J Am Coll Cardiol. 2012 Jun 5;59(23):2091-8
5. Than et al. A 2-h diagnostic protocol to assess patients with chest pain symptoms in the Asia-Pacific region (ASPECT): a prospective observational validation study. Lancet. 2011 Mar 26;377(9771):1077-84
6. Cullen et al. Validation of High-Sensitivity Troponin I in a 2-Hour Diagnostic Strategy to Assess 30-Day Outcomes in Emergency Department Patients With Possible Acute Coronary Syndrome. Journal of the American College of Cardiology, Volume 62, Issue 14, 1 October 2013, Pages 1242-1249
7. Antman EM, Cohen M, Bernink PM, et al. The TIMI Risk Score for Unstable Angina/Non–ST Elevation MI: A Method for Prognostication and Therapeutic Decision Making. JAMA. 2000;284(7):835-842
8. Chase et al. Prospective Validation of the Thrombolysis in Myocardial Infarction Risk Score in the Emergency Department Chest Pain Population. Annals of Emergency Medicine September 2006 (Vol. 48, Issue 3, Pages 252-259
9 . Cullen et al. Validation of High-Sensitivity Troponin I in a 2-Hour Diagnostic Strategy to Assess 30-Day Outcomes in Emergency Department Patients With Possible Acute Coronary Syndrome. Journal of the American College of Cardiology, Volume 62, Issue 14, 1 October 2013, Supplementary appendix
10.Pope et al. Missed Diagnoses of Acute Cardiac Ischemia in the Emergency Department. N Engl J Med 2000; 342:1163-1170 April 20, 2000
11. Mitchell et al. Prospective multicenter study of quantitative pretest probability assessment to exclude acute coronary syndrome for patients evaluated in emergency department chest pain units. Ann Emerg Med. 2006 May;47(5):447.
12. Kline et al. Outcomes and Radiation Exposure of Emergency Department Patients With Chest Pain and Shortness of Breath and Ultralow Pretest Probability: A Multicenter Study. Annals of Emergency Medicine – 14 October 2013
- EM Nerd-The Case of the Partial Cohort - May 24, 2020
- EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
- EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019
Hey
Great discussion regarding the ADAPT and ASPECT trials. I am a medical student at the University Of Queensland and currently looking at the data from that trial, specifically the physiological parameters (heart rate, BP, temp., edema etc) at presentation and the 30 day outcomes of such patients. Will let you know the results–I think this sort of analysis is what you are referring to when you said we should be focussed on ” studies validating our clinical exam with real patient oriented outcomes ”
Cheers,
Daniel Beamish
That would be fantastic! It would be fascinating to see the patient level data. Thanks for the feedback I really appreciate it!
Nice job pointing out the futility of over utilization of testing in chest pain patients. I have read several of your chronicles and agree with the “art of doing nothing” when appropriate which is much more often than commonly appreciated. Striving for diagnostic perfection, while a laudable goal, is frequently impractical and harmful. While continuing to embrace technology we must utilize our imperfect clinical judgement and somehow convince our peers and society that doing everything for everybody is in no one’s best interest, as Greg Henry’s philosophical rants on this topic so astutely point out. I am very impressed that… Read more »
Thanks Chris for you kind remarks. These challenges, I suspect, will only get worse as medical technology progresses and becomes even more accessible. We so often fall into the trap that “more information is always better.”
Thanks again and I will do my best not to disappoint in the future!