We have discussed the dangers of surrogate outcomes at length, but none are more evident to an Emergency Physician than the time-based metrics we are subjected to on a daily basis. The latest of these temporal surrogates forced upon us is the 3-hour bundle of care in patients presenting to the Emergency Department with symptoms concerning for sepsis. In a recent article published in NEJM by Seymour et al, the utility of this time-based surrogate measure was placed in its appropriate clinical context (1).

The authors performed a retrospective analysis of 185 hospitals involved in the New York State Department of Health sepsis registry database. These authors were not involved in the clinical implementation or curation of this immense undertaking, which prospectively collected data on patients presenting to the Emergency Department with sepsis or septic shock. Patients were included in this analysis if they were treated according to a pre-specified 3-hour treatment bundle within 6 hours of presentation to the Emergency Department. While the bundles varied from hospital to hospital, all participating centers were required to include blood cultures before the administration of antibiotics, measurement of the serum lactate level, and the administration of broad-spectrum antibiotics. The authors included patients age 17 and older who had either severe sepsis or septic shock as defined by the 2001 International Sepsis Definitions. The authors excluded patients in whom the 3-hour bundle took greater than 12 hours to complete in the hopes of decreasing the introduction of bias due to treatment outliers.

The authors examined 49,331 patients seen at 149 Emergency Departments from April 2014 to June 2016. The median time to complete the 3-hour bundle was 1.30 hours, the median time to administer broad-spectrum antibiotics was 0.95 hours, and the median time to complete the initial fluid bolus (30 cc/kg) was 2.56. Using an internally derived and validated risk model the authors performed a multivariable regression and determined each hour required to complete the 3-hour bundle was associated with a statistically significant increase in mortality (odds ratio of 1.04/hour 95%-CI 1.02-1.05; p<0.001). This same association was seen with the time to administration of broad spectrum antibiotics (odds ratio 1.04/hour; 95% CI, 1.03 to 1.06; P<0.001), time to completion of blood cultures (odds ratio 1.04 per hour; 95% CI, 1.02 to 1.06; P<0.001), and time to obtaining serum lactate (1.04 per hour; 95% confidence interval, 1.02 to 1.06; P<0.001), the latter two presumably serving as surrogates for the prior. This association was not demonstrated when the time to completion of fluid bolus was examined in isolation. The odds ratio for mortality was 1.01/hour; 95% CI, 0.99 to 1.02; P=0.21.

And while this was a well done retrospective analysis and the results likely represent the importance of early administration of antibiotics in patients with sepsis and septic shock, I cannot help but cringe at the potential effects this data will have on Emergency Department care. First let us remember this is a retrospective analysis and we simply cannot determine the causative effect of timely completion of the 3-hour bundle. Hershey et al (2) illustrate a progressive decrease in sepsis related mortality over time that is associated with the institution of this mandated bundle in New York state. But they go on to note sepsis related mortality has been falling over time nationwide, with or without mandatory bundled care.

Let us for a moment assume that the results presented by Seymour et al are causative. How exactly do we quantify the clinical benefits of bundled care? The authors found that failure to complete the 3-hour bundle increased mortality in a statistically significant fashion. The odds ratio was 1.04. Which essentially means delays in the completion of the 3-hour bundle would have to occur in 148 patients before one dies or a NNH of 148 (3). In fact, if you examine the crude inpatient mortality between the patients who had the 3-hour bundle completed in under 3 hours compared to those in whom it was completed between 3-12 hours, they differ very little (22.6% vs 23.6%). It is important to keep in mind this is a very select population. Seymour et al did not enroll all patients with sepsis, but rather examined only the subset of patients with severe sepsis or septic shock as per the Sepsis-2 criteria. In fact, one could argue even this criteria was too broad as the small improvements in mortality with early aggressive care vanished when you examine the less critically ill subset of this cohort. The odds ratio for inpatient mortality, when the patients requiring vasopressors are excluded from the analysis, was 1.02/hour (95% CI 1.00-1.03).

In 2000 JAMA published data from a prospectively gathered registry of 27,080 patients with ST elevation myocardial infarctions undergoing primary PCI (4). Using multivariable logistic regression, Cannon et al calculated mortality rates at 30-minute intervals and reported a statistically significant increase in mortality when delay to PCI was longer than 120 minutes. And despite the potential confounding associated with such an analysis, this document was used to justify the widespread utilization of door-to-balloon time as the metric to determine quality of care in STEMI patients nationwide. So it should come as no surprise when in 2013 Menees et al reported on door-to-balloon-times over a 4 year period, finding a decrease from an average of 82 minutes to 67 minutes resulted in no change in overall mortality (5). In 2015 Fanari et al reported on the results of an aggressive protocol meant to improve door-to-balloon times (6). They found that despite reducing door-to-balloon time by 15 minutes, they found no improvement in mortality. Moreover, the rate of false positive STEMIs increased from 7.7% to 16.5%, and in the patients brought to cath lab in error the mortality rose from 5.6% to 21.7%. This is not just an archaic fable warning against the misuse of surrogate endpoints, but rather a day to day reality of the logistical chaos suffered upon us when one of these surrogates is applied in a mindless fashion.

I do not think anyone would argue the timely administration of broad spectrum antibiotics and aggressive fluid resuscitation are both vital to the management of patients presenting to the Emergency Department with sepsis or septic shock. But this represents only a small minority of patients who will be affected by the blind application of such time dependent performance metrics (7). Rather studies such as Seymour et al will be used in vain to justify the widespread and reckless application of these strategies to a broad cohort of Emergency Department patients. Like door-to-balloon time in STEMI and door-to-needle time in stroke, all nuances and caveats surrounding this surrogate will be forgotten, and all that will be remembered is a clinically vague odds ratio and corresponding p-value. When applied broadly its application will be unhelpful in most, harmful in some, but most importantly logistically cumbersome to the already overburdened Emergency Department.

Sources Cited:

1. Seymour CW et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. NEJM, May 21, 2017
2. Hershey TB et al State Sepsis Mandates — A New Era for Regulation of Hospital Quality. NEJM, May 21, 2017
3. Sackett, Jonathan J. Deeks and Doughs G. Altman. Down with odds ratios! David L. Evid Based Med 1996 1: 164-166.
4. Cannon CP, Gibson CM, Lambrew CT, et al. Relationship of symptom-onset-to-balloon time and door-to-balloon time with mortality in patients undergoing angioplasty for acute myocardial infarction. JAMA. 2000; 283: 2941–2947.
5. Menees DS et al. Door-to-balloon time and mortality among patients undergoing primary PCI. N Engl J Med. 2013 Sep 5;369(10):901-9.
6. Fanari Z, Abraham N, Kolm P, et al. Aggressive Measures to Decrease “Door to Balloon” Time and Incidence of Unnecessary Cardiac Catheterization: Potential Risks and Role of Quality Improvement. Mayo Clin Proc. 2015;90(12):1614-22.
7. Singer M. Antibiotics for Sepsis – Does Each Hour Really Count? Or is it Incestuous Amplification? Am J Respir Crit Care Med. 2017.

Additional FOAM resources:

SGEM 181: Did You Ever Have to Make Up Your Mind, Pan Scan or Leave Other Scans Behind?

The Bottom Line: REACT-2

St.Emlyn’s: Do we always need a whole body CT in trauma?

• Author
• Recent Posts

Rory Spiegel

EM and Critical Care Attending
University of Georgetown
Resuscitation and Critical Care Fellowship Graduate

Creator
EMNerd.com

Latest posts by Rory Spiegel (see all)

• EM Nerd-The Case of the Partial Cohort - May 24, 2020
• EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
• EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019