Introduction

A satisfactory clinical definition of sepsis has been eluding us since the ancient Greeks first coined the term.  Current definitions of sepsis attempt to achieve two goals:  to provide a rapid screening test and to render a definitive diagnosis.  However, it remains unclear whether any definition can achieve this.

Re-defining syndromes creates exciting headlines, but it is unclear if it improves patient care.  This post explores some problems with the new sepsis definition.  If Sepsis-III is adopted into clinical practice, it will be important to understand its limitations.

Problem #1: Sepsis-III remains subjective

In all sepsis definitions, the diagnosis of sepsis hinges on whether the patient has “suspected infection.”  Unfortunately, there is little discussion about exactly how to determine whether infection is “suspected.”  Indeed, for most patients presenting with an unclear problem, infection is somewhere on the differential diagnosis.

Sepsis-III recognizes the universal possibility of infection in undifferentiated patients.  As highlighted in red (adjacent box), the guidelines recommend that infection be suspected in any patient with unexplained dysfunction of any organ (a description which matches most critically ill patients in the emergency department).  Defining “suspected infection” so broadly renders this criterion nearly meaningless.

Problem #2: qSOFA & SOFA are mortality predictors, not tests for sepsis

The SOFA score is an illness-severity score which may be used to predict the mortality of any critically ill patient.  qSOFA was also designed to predict mortality (albeit within the context of a cohort of patients with suspected infection).  Thus, qSOFA and SOFA are predictors of mortality; they are not tests of sepsis.

Unfortunately, qSOFA will inevitably be misunderstood to be a “sepsis screen.”  This may lead to a dangerous assumption that anyone with positive qSOFA must be treated for sepsis, whereas they actually require a thoughtful investigation for a variety of severe illnesses (e.g. cardiogenic shock, pulmonary embolism, etc.).  Similar errors are common when interpreting an elevated lactate, which is often misconstrued as a test for sepsis (1).

Problem #3: Sepsis-III is less specific for infection than Sepsis-II

Both components of the Sepsis-II definition provided some degree of selection for patients with infection.  In contrast, the only component of Sepsis-III which selects for patients with infection is the [suspected infection] component.  Since Sepsis-III has fewer elements that select for infection, it should have a lower specificity than Sepsis-II for selecting infected patients (versus critically ill, non-infected patients).

The specificity of Sepsis-III for infection is unknown because it hasn't been prospectively validated.  In practice, it is likely that when physicians encounter a patient who is qSOFA-positive, this would elevate their index of suspicion for infection (as recommended in the guidelines: highlighted red text above).  This may create a pathway of circular logic which leads directly from qSOFA to a provisional diagnosis of sepsis.

Problem #4: qSOFA has similar performance compared to SIRS for mortality prediction

Currently the SIRS criteria have been criticized for being poorly sensitive and are increasingly unpopular.  Thus, one of the decisions by the Sepsis-III taskforce was to replace SIRS with qSOFA.  However, the performance of qSOFA for mortality prediction is similar to the SIRS criteria (2):

SIRS and qSOFA have similar areas under the ROC curve, indicating that they have a similar overall test performance (3).  qSOFA has a higher specificity than SIRS, but this comes at the cost of a lower sensitivity.  Thus, qSOFA isn’t substantially better than SIRS, it has merely traded sensitivity for specificity.  The ROC curves for SIRS and qSOFA unfortunately weren't published, but one might imagine that they look something like this.

Problem #5: qSOFA may be less specific in diseases that directly cause hypotension, tachypnea, or delirium

An abnormality in any qSOFA criteria could be roughly conceptualized as one of the following:

• Spurious abnormality:  little to do with the active disease process.
• Primary abnormality:  local consequence of disease process which primarily affects that organ.
• Secondary abnormality:  remote consequence of a disease process primarily affecting a different organ.

Examples of these types of abnormalities include:

Any abnormality in the qSOFA criteria is concerning.  However, spurious abnormalities are the least concerning (e.g. a demented patient with a splinter in their foot has “altered mentation”).  Primary abnormalities are more worrisome, but they may still carry a fairly good prognosis (e.g. asthmatic patient with respiratory rate of 23, unruly drunk with altered mentation).  Secondary abnormalities are generally most concerning, because they indicate a disease process severe enough to affect the function of a distal organ (e.g. patient with hemorrhagic shock causing confusion).

qSOFA may be less specific for diseases involving the heart, lungs, or brain, which directly cause hypotension, delirium, or tachypnea.  In such cases, the patient will often have one qSOFA criteria as a direct consequence of their disease.  For example, an unruly drunk walks into the emergency department with one qSOFA criteria primarily due to alcohol intoxication.  All it takes is one more criteria to make him qSOFA-positive (e.g. tachypnea due to agitation).

Problem #6: qSOFA is inconsistent with a validated prognostic model (CURB65)

Pneumonia is the most common source of septic shock.  This naturally leads to a comparison of qSOFA with the CURB65 prognostic score for pneumonia.

Comparison suggests that qSOFA over-estimates the mortality of a pneumonia patients.  For example, consider a 50-year-old patient with pneumonia, respiratory rate of 24 breaths/minute, blood pressure of 95/65, BUN of 15 mg/dL, and normal mental status.  According to qSOFA, this patient has sepsis and perhaps should be transferred to the ICU.  However, according to the CURB65 score, this patient has a 0.6% mortality and could be sent home with oral antibiotics.

This is an example of how qSOFA may be less specific in patients with primary pulmonary disease (as discussed above).  A respiratory rate >22 is not particularly unusual or alarming in a patient with pneumonia (4).  Thus, many pneumonia patients will start out with one qSOFA criteria, only requiring one additional criteria to become qSOFA-positive.

Problem #7: Combining qSOFA and SOFA scores is not evidence-based among patients outside the ICU

A common approach to diagnosis is shown here.  When a condition is suspected (e.g. pulmonary embolism), first a screening test is utilized.  An ideal screening test has a high sensitivity, but is fast and easy to perform (e.g. D-dimer).  Patients who have a positive screening test will need to proceed onward to a definitive test, which is often more expensive or difficult to perform, but has both high sensitivity and specificity (e.g. CT angiogram).

The sepsis diagnostic algorithm seems to be designed with qSOFA as a screening test and SOFA as a definitive test.  On face value this makes sense, because qSOFA is a simple and fast, whereas the full SOFA test is labor intensive.

However, the specificity of SOFA is actually lower than the specificity of qSOFA, making this test sequence illogical.  Thus, SOFA adds little to qSOFA among patients outside the ICU (5).

Problem #8: The combined performance of {qSOFA + SOFA} for mortality is not reported.

Although evidence is provided regarding the performance of qSOFA and SOFA, there is no evidence provided about the performance of the combination of {qSOFA+SOFA}.  Since qSOFA and SOFA scores are not statistically independent, it is difficult to predict how they will function in combination.  To explore this, lets imagine two extreme possibilities:  qSOFA and SOFA either being maximally concordant or maximally discordant:

Discordance improves the specificity of the combined test sequence, because only one test needs to be negative to exclude sepsis.  However, discordance impairs the sensitivity, because both tests must be positive to rule in sepsis.  The tests are almost certainly more concordant than discordant.  Without evidence, the only definitive conclusion is that the combined tests have a sensitivity between 23-55% and a specificity between 84-100% (2).

Problem #9: The overall sensitivity of Sepsis-III for sepsis might be <50% outside of the ICU

As discussed above the sensitivity of {qSOFA+SOFA} for mortality is likely <55%.  However, the most important patients to identify are patients who receive critical care and subsequently survive (i.e. truly benefit from their ICU care).  These ICU survivors may initially look less ill than the patients who die.  Therefore, the sensitivity of {qSOFA+SOFA} for ICU survivors is likely lower than for nonsurvivors (i.e., <<55%).

Finally, the sensitivity of the “suspected infection” criteria is unknown (but almost certainly below 100%).  Taking all of these factors into account, the sensitivity of Sepsis-III criteria for sepsis could be under 50% (6).

Problem #10: Sepsis-III is not a consensus guideline in the United States

Sepsis-III has been endorsed by the Society of Critical Care Medicine, the American Thoracic Society, and the American Association of Critical Care Nurses.  However, it has not been endorsed by the American College of Chest Physicians, the Infectious Disease Society of America, any of the Emergency Medicine societies, or any of the hospital medicine societies.  It is difficult to call this a consensus guideline without support from Emergency Physicians or Hospitalists, who diagnose sepsis most often.

[Update: since posting this, the American College of Chest Physicians has issued a strong statement opposing Sepsis III]

• Sepsis-III remains a subjective definition, with little guidance regarding what exactly is meant by “suspected infection.”
• qSOFA and SOFA are predictors of mortality, they are not tests of sepsis.
• Sepsis-III is less specific for infection compared to Sepsis-II. The only thing that differentiates Sepsis-III from a screening test to identify sicker patients is the “suspected infection” component.
• Compared to SIRS, qSOFA is more specific (but less sensitive) for predicting mortality.
• Based on available retrospective data, the sensitivity of Sepsis-III for sepsis is <55% among patients outside of the ICU.
• Prospective validation is needed to determine the real-world performance of Sepsis-III.

Notes

1. Misunderstanding of qSOFA isn’t technically a failure of Sepsis-III, but rather our failure to correctly apply the definition.  Nonetheless, qSOFA and SOFA components are emphasized in Sepsis-III (with scant discussion of the “suspected infection” component).  Thus, this may be an easy mistake to make.
2. All evidence discussed here pertains to patients outside the ICU.  The primary clinical utility of sepsis definitions is to determine who is sick and who needs to be admitted to the ICU.  Sensitivity and specificity values are obtained from eTable 3 in the supplemental data accompanying Seymour et al.  Please note that there appears to be a discrepancy between the text of the article and the table regarding the performance of the SIRS criteria, with data shown here being based on eTable 3 (The text states that for patients outside the ICU, “55% of decedents had 2 or more SIRS criteria, whereas 81% of survivors had less than 2 SIRS criteria” – this is inconsistent with the eTable 3 which shows values of 64% and 65%, respectively).
3. A receiver-operator curve (ROC curve) is a graph of the sensitivity vs. specificity of a test at every possible value of the test.  This is a useful method to compare the overall performance of two tests, independent of any specific cutoff.  A perfect test would have an area under the ROC curve of 1.0, whereas a completely worthless test would have an area of 0.5.  However, once you've selected a cutoff point, the area under the ROC curve is less relevant than the sensitivity and specificity of the cutoff that has been selected.
4. Respiratory rate clearly has enormous prognostic value in pneumonia, but the cutoff of 22 b/m is too low in this disease process.  I usually start getting substantially more worried when the respiratory rate is above 30, consistent with the CURB65 score.
5. SOFA performs better among ICU patients.  However, clinically the definition of sepsis is mostly useful outside the ICU (e.g. identifying patients who require ICU transfer).
6. The fact that the “definition” of sepsis may not capture most patients who require ICU care for infection management creates some strange linguistic problems.  I suppose that if we accept the Sepsis-III definition then, by definition, it must have a performance of 100%?

Related materials

• Sepsis-III
  • Rebel EM
  • FOAMcast
  • St Elmyn's
  • First 10 EM
  • JAMAsepsis.com – tons of resources on Sepsis-III
• Statements from groups opposing Sepsis-III
  • American College of Chest Physicians (ACCP)
  • UK sepsis trust
  • Latin American Sepsis Institute
  • (will attempt to keep this list up-to-date… likely more on the way)
• Septic shock discussion at SMACC
  • SMACC sepsis panel discussion
  • Preemptive sepsis panel SmaccBack (EmCrit)
• PulmCrit sepsis stuff:
  • Overall concept: Renoresuscitation & avoidance of chronic sepsis
  • Overall strategy: Accelerated goal directed therapy
  • Norepinephrine…early to stabilize MAP
  • Vasopressin… & renal microvascular hemodynamics
  • Epinephrine… & the significance of lactate
  • Stress-dose steroids: 4 misconceptions & 1 truth

• Author
• Recent Posts

Social Me

Josh Farkas

Josh is the creator of PulmCrit.org. He is an associate professor of Pulmonary and Critical Care Medicine at the University of Vermont.

Social Me

Latest posts by Josh Farkas (see all)

• PulmCrit: ADAPT and SCREEN trials are full of sound and fury, signifying little - December 13, 2024
• PulmCrit: How to quickly create a useful professional account in BlueSky - November 28, 2024
• PulmCrit Wee: Why MedTwitter should move to Bluesky - November 15, 2024