On the morning of January 1st, 2017 a number of proud South Carolinians woke up to find themselves no longer living in their former state of residence. Without moving from the beds in which they slept, they were all magically transported into the neighboring the state of North Carolina. Due to a border dispute which has been underway for more than 20 years, the state line which separated these two proud states was moved approximately 12 miles north of its previously designated coordinates. And while to many of us this seems like amusing trivia, to the proud South Carolinians it directly affected, the consequences likely appeared more serious.
The same sort of shifting in geographic territories happened in the world of sepsis, in February 2016 when Singer et al published the new Sepsis 3.0 definitions (1). Not only was this a change in the current thinking of the underlying mechanisms causing this deadly presentation, but also a change in the clinical tools suggested to identify high risk patients.
The authors defined sepsis as “life-threatening organ dysfunction caused by a dysregulated host response to infection”. And while this definition is likely a better representation of the physiologic underpinnings of the disease in question, it does little to change the day-to-day logistical management of the patients that present to us in the Emergency Department. The authors go on to suggest a change in how we identify sepsis and septic shock in the clinical arena. Sepsis is defined as an infection associated with an increase in the Sequential Organ Failure Assessment (SOFA) score of 2 points or more, and septic shock as fluid resistant shock requiring vasopressor supplementation to maintain a mean arterial pressure of 65 mm Hg or greater and serum lactate level greater than 2 mmol/L.
More importantly, for the Emergency Department, the authors developed a novel clinical score to be used outside of the ICU to identify patients with sepsis or septic shock. Calling it the quick SOFA (qSOFA), this score consisted of a respiratory rate >22, altered mental status, and hypotension (a SBP < 100 mm Hg). In the original manuscript deriving and validating this score, Seymour et al noted its moderate predictive value outside of the ICU setting. Using the UPMC medical care systems records these authors retrospectively identified patients who received antibiotics and had body fluid cultures drawn during their stay in the hospital. The authors required these two events to occur within 24-hours of each other. They then derived and validated the qSOFA, using the traditional SIRS criteria and the more data laden SOFA score as comparators (2).
In this original derivation cohort, the qSOFA performed admirably when utilized in a population outside the ICU setting, with an area under the curve (AUC) of 0.81. This was compared to 0.76, and 0.79 for SIRS and SOFA respectively. When validated in multiple external data sets, the qSOFA’s performance remained consistently acceptable (0.71, 0.75, 0.78 respectively). A great deal has been written on the utility of this score, essentially stating that given the retrospective origin of its derivation and validation, it is difficult to determine how it will perform in the Emergency Department when used in a prospective fashion. A number of retrospective studies have attempted to validate qSOFA’s performance and have found variable diagnostic characteristics, but like their predecessor, they suffer from the same retrospective, big data dredge methodology that leads to uncertainty regarding the scores stability (3,4).
Published in JAMA, almost one year after Seymor et al unveiled this controversial tool, Freund et al reported the first prospective analysis of the qSOFA score (5). These authors examined patients presenting to 30 Emergency Departments across Europe with suspected infection. Patients were only included in the final analysis if two “experts” reviewing all the pertinent data from the patients’ hospital course concluded the hospitalization was due to an infectious cause. The 3 components of the qSOFA score were collected prospectively by the treating Emergency Physician during their ED course. These values were not collected exclusively upon initial presentation, but rather utilized the patient’s worst score during their stay in the Emergency Department. Unlike the previous retrospective evaluations, the presence of altered mental status was recorded separately from the patient’s GCS. The components of the SIRS criteria and SOFA score were also collected during their ED stay.
The authors sought to compare a qSOFA score of 2 or greater, an increase in the SOFA score by two points, 2 or more SIRS criteria, and severe sepsis, which was defined as 2 or more SIRS criteria and a lactate > 2 mmol/L. They examined the diagnostic accuracy of each of these decision tools to identify patients who would die during their hospital course. They also examined admission to the ICU, length of stay in the ICU for greater than 72-hours and a composite of in-hospital mortality and 72-hour ICU admission.
Of the 1088 patients screened, 879 were included in the final analysis. At a glance the qSOFA performed admirably. The highest AUC for predicting in-hospital mortality was achieved by the qSOFA at 0.80. This was followed by SOFA (0.77), SIRS (0.65) and severe sepsis (0.65). The authors reported the findings of the multiple secondary endpoints they measured closely resembled the results of their primary analysis. And so, in a sense Freund et al validated Seymour et al’s original derivation and validation cohort. But of course the reality is always far more complex.
While qSOFA may have outperformed SIRS, SOFA and severe sepsis, none of these scores performed that well. qSOFA which had the best diagnostic test characteristics out of the scores examined, only had a sensitivity of 70% and a specificity of 79%. I am not sure if any of us would be comfortable missing 30% of the patients who died from sepsis.
But in truth, none of these scores should be utilized in any typical diagnostic fashion. After all, our goal in the Emergency Department should not be to predict who will perish during their hospital course, but rather to identify a population of critically ill patients that will benefit from the aggressive resuscitative measures we have come to associate with the management of sepsis and septic shock. And while this cohort is far more nebulous, examining each of these tools prognostic abilities gives us a better idea of their respective performances.
In the Freund et al cohort the overall mortality was reported to be 8%. For patients with a qSOFA score less than 2, the in-hospital mortality was only 3%. Conversely, in the patients who had a qSOFA of 2 or greater, the in-hospital mortality was 24%. In contrast, for the patients with less than 2 SIRS criteria the in–hospital mortality was fairly similar to those in the low risk qSOFA group (2.2%), but the in-hospital mortality of patients with 2 or more SIRS criteria did not differ significantly from the overall mortality of the entire cohort (8% vs 10.6%). This suggests while the SIRS criteria can identify a cohort at very low risk for death during their hospital course, it does so in such a small minority it is essentially clinically useless. In fact, of the 879 patients in the final analysis 653 (74%) had 2 or more SIRS criteria. In contrast, the qSOFA score only included 218 (25%) of the patients in its high risk cohort. Furthermore, this analysis likely overestimates the SIRS criteria’s poor specificity. Since they only examined the patients who were admitted to the hospital for suspected infection, they excluded all patients with a benign infection that commonly present to the Emergency Department with 2 or more SIRS criteria and are discharged home.
There is an even more important issue to discuss. Let us for a moment look at the components of the qSOFA score. Altered mental status, hypotension (defined as a systolic blood pressure less than 100 mm Hg), and an increased respiratory rate (greater than 22 breaths/minute). In order to achieve a score of two or more a patient has to be hypotensive and altered, hypotensive and in respiratory distress, or be altered and in respiratory distress. None of these patients are clinical subtle. qSOFA is not a marker of sepsis, rather it is a marker for severity of illness. A recent retrospective analysis by Singer et al, published in the Annals of Emergency Medicine, demonstrated the qSOFA score performed equally well (or poor) in patients with and without infectious causes of their presentation (4). And so, if we are to believe the results presented by Freund et al, then for the most part sick patients look sick. While some patients that appear well in the Emergency Department will go on to decline during their hospital course, the majority of patients presenting to the Emergency Department with sepsis do so in a clinically obvious fashion.
So the next question becomes, “Is there a method to identify the remainder of these patients who are secretly harboring a critical illness and will go unnoticed in the Emergency Department?”. The first solution is to utilize a tool like the SIRS criteria that has a far higher sensitivity the qSOFA. But it achieves this superior sensitivity at the cost of including the majority of patients presenting to the Emergency Department with signs of infection. Adding more hay to the stack does not make it easier to identify the few needles that would otherwise go unnoticed. Lactate has been suggested as a tool to identify this clinically occult subset of patients. But both Seymour et al’s original data set and Freund et al’s recent validation set would suggest the addition of lactate adds nothing to the risk stratification the qSOFA score already provides. Seymour et al found adding lactate at a threshold of 2, 3 or 4 mmol/L to the qSOFA score did not augment its diagnostic prowess. In Freund et al’s recent analysis when the authors added lactate to the qSOFA score, it did not change its prognostic value.
These findings should actually not come as a surprise. The original data on lactate supports its use in the identification of cryptic shock, or tissue hypoxia, without outward signs of shock. In the seminal paper published in Intensive Care Medicine in 2007 by Howell et al, the authors conducted a prospective observation trial which enrolled all patients who presented to their Emergency Department with a suspected infection (6). The authors obtained serum lactate levels in all patients and examined its ability to predict in-hospital mortality. They found patients with increased serum lactate values died at a more frequent rate than patients with numerically normal values. Serum lactate still maintained its predictive capability mortality even when the authors controlled for hypotension. Of the 1287 patients enrolled, 73 (5.7%) died during their hospital course. Patients who were neither hypotensive nor demonstrated an elevated lactate (> 4 mmol/L) had an in-hospital mortality of 2.5%. Conversely in the patients who were hypotensive or had an elevated lactate, the mortality was 28.3%. When you compare the Howell et al and Freund et al cohorts side by side the predictive values of qSOFA and lactate appear fairly similar. Puskarich et al in their reanalysis of the Jones trial found fairly similar results to Howell’s original assessment of lactate’s predictive abilities. They found that a serum lactate was an equally good predictor of mortality as hypotension. The respective in-hospital mortality rates in these cohorts were 21% and 19% (7).
Clinically occult shock (a patient with a normal BP) should not be misinterpreted as a clinically occult patient (a well appearing patient). In Puskarich et al, although the cryptic shock patients were not hypotensive in the strictest sense, they were by no means physiologically normal. On the contrary they were older, more tachycardic, with faster respiratory rates, than their hypotensive counterparts. And though they were not hypotensive (<90 mmHg), their blood pressures were not necessarily normal. The median blood pressure in the cryptic shock group was 108mmHG with an IQR of 92-126 (7). To put it simply, these patients were clinically ill appearing and likely would have been identified using the qSOFA criteria. In fact, while Howell et al found a lactate threshold of 4 mmol/L, to be very specific (92%) it only identified a small minority of the patients who went on to die during their hospital stay (sensitivity of 36%). The analyses by Seymour et al and Freund et al would suggest this subset of critically ill patients identified by an elevated serum lactate is included in the cohort identified with qSOFA score of 2 or greater. And so while lactate still serves a purpose to identify the spectrum of patients who are critically ill, I doubt it will help us recognize the patients who present to the Emergency Department well appearing but go on to decompensate during their hospital course.
The true complexity of sepsis is still as of yet being uncovered. What we do know is it presents itself in a multitude of fashions, and its clinical course and the rapidity of its progression is determined by more variables than we are even aware. To think we can identify a set of clinical characteristics that is both easily utilized at the bedside and will perfectly differentiate the subset of critically ill patients from the more clinical benign is at best naive. And while qSOFA and the SIRS criteria are both reasonable from a triage perspective (depending on your accepted signal vs noise ratio), neither will stand up to the clinical judgment of an experienced Emergency Physician doing what he or she is trained to do, to differentiate the sick from the not sick. It is also quite possible that we will never identify a tool that detects all the patients at risk of decompensation during their hospital stay because there exists a subset of patients for which clinical decline is not predictable at the time of initial presentation. At least not without adding so much diagnostic noise it drowns out any clinically useful signal. In the end, sepsis and septic shock are clinical diagnoses, far more nuanced than any criteria can encapsulate. Sepsis 3.0 may very well have changed the geographic borders in which we reside, but here in the trenches of the Emergency Department it is still the same rocky ground below our feet.
- Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810.
- Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8): 762-774.
- Churpek MM, Snyder A, Han X, et al. qSOFA, SIRS, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients Outside the ICU. Am J Respir Crit Care Med. 2016;
- Singer, Adam J. et al. Quick SOFA Scores Predict Mortality in Adult Emergency Department Patients With and Without Suspected Infection Annals of Emergency Medicine
- Freund Y et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA January 17, 2017 Volume 317, Number 3
- Howell MD, Donnino M, Clardy P, Talmor D, Shapiro NI. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Med. 2007;33(11):1892-9.
- Puskarich MA, Trzeciak S, Shapiro NI, et al. Outcomes of patients undergoing early sepsis resuscitation for cryptic shock compared with overt shock. Resuscitation. 2011;82(10):1289-93.
University of Maryland
Resuscitation Fellowship Graduate
Latest posts by Rory Spiegel (see all)
- EM Nerd-The Case of the Deceptive Beacon - May 25, 2018
- EM Nerd-The Peculiar Case of the Parallel Pathways - May 18, 2018
- EM Nerd-The Case of the Temporal Fallacy - April 26, 2018