Sepsis 3.0 replaced the SIRS criteria with a new risk-stratification tool, qSOFA. qSOFA was initially developed within the Sepsis-3 publication itself. Until now, qSOFA has never been validated. The value of qSOFA vs. SIRS remains controversial.
Churpek 2016: qSOFA, SIRS, and early warning scores for detecting clinical deterioration in infected patients outside the ICU.
This was a study of 30,677 patients in the emergency department and ward at the University of Chicago who were suspected of having infection (defined as any anyone cultured and started on IV antibiotics). Electronic records were retrospectively analyzed to calculate SIRS, qSOFA, and two risk-stratification scores (MEWS and NEWS). These scores were compared to a primary outcome of in-hospital mortality and a combined outcome of mortality or ICU admission.
MEWS and NEWS are risk-stratification scores, designed and validated to identify patients at risk for deterioration (tables below). They are fairly similar, with NEWS being newer:
The overall performance of these tests is best reflected by the area under the receiver-operator curve. NEWS consistently came out ahead. In predicting mortality or ICU transfer (arguably the most relevant outcome), qSOFA and SIRS were similar (1):
The figure below shows the test sensitivity versus percent of patients with a positive screen for various cutoffs of different tests, using the combined outcome (mortality or ICU transfer). qSOFA and SIRS have similar overall test performance, the main difference being that SIRS is more sensitive whereas qSOFA is more specific. MEWS and NEWS have superior performance:
This study focused mostly on the highest test score before ICU transfer, rather than the test score at the point in time when infection was first suspected. This data won't apply to the bedside clinician who is interpreting the test score at a single time-point. For example, the sensitivity of a single test score will be lower than the sensitivity of the worst score before ICU transfer.
This figure explores when these indices turned positive. The cumulative likelihood that a patient would meet various criteria at any point in their hospitalization is shown, prior to the time of death or ICU transfer. The sensitivity of qSOFA is poor, especially >12 hours before deterioration (<40%).
These results shouldn't be too surprising
This study suggests that compared to SIRS, qSOFA has increased specificity at cost of decreased sensitivity. Both tests have similar global performance. This was previously discussed on this blog and comes as no surprise:
This study also shows that the NEWS score out-performs qSOFA. qSOFA is a rudimentary risk-stratification score, so it isn't surprising that it would be out-performed by more sophisticated risk-stratification scores (e.g. APACHE-II). NEWS is a more detailed score than qSOFA, which integrates a greater amount of vital information:
Indeed, if you look very carefully at the NEWS score, it actually contains the components of the qSOFA score (figure below). Since qSOFA is effectively a pared-down version of the NEWS score, it makes sense that qSOFA shouldn't perform as well.
qSOFA plays a central role in the Sepsis 3.0 definition:
- qSOFA has a poor sensitivity
- qSOFA is a late indicator of deterioration
- qSOFA is inferior to the NEWS score (despite the NEWS score being based on data which is equally easy to obtain at the bedside)
Sepsis-3 still hasn't been widely adopted into hospital protocols or Medicare guidelines. Widespread change of such protocols requires an enormous expenditure of time and money (e.g. re-writing policies, educating staff). Such change can only be justified by a definition which has been successfully validated. Failure of qSOFA to be validated suggests that additional evidence is required before considering adoption of Sepsis-3.
Should we follow the lead of our colleagues in the UK?
These organizations don't recommend qSOFA as a primary evaluation for sepsis. Instead, the screening test for sepsis in the UK is the NEWS score. They may be far ahead of us on this one. Notably, the NEWS score has been successfully validated and deployed on a large scale.
If the NEWS score is used, it should be understood that it is not a test for sepsis. NEWS is a global risk-stratification tool which identifies patients who are critically ill from any disease. Thus, an elevated NEWS score should prompt a thoughtful evaluation for any potential life-threat. Increased utilization of risk-stratification tools such as NEWS could facilitate early recognition of sepsis as well as other critical illnesses (cardiogenic shock, pulmonary embolism, hemorrhage, etc.).
- Churpek et al. 2016 is the first study to attempt validation of qSOFA.
- qSOFA and SIRS have similar overall performance in predicting the combined outcome of death or ICU transfer. qSOFA has a higher specificity, but this comes as a tradeoff for lower sensitivity.
- qSOFA is <40% sensitive for detecting a patient who will die or need ICU transfer in 12 hours.
- qSOFA is consistently out-performed by the NEWS score, a more sophisticated bedside risk-stratification tool (figure below).
- This study doesn't support the Sepsis-3 definition. Further evidence is needed before considering the widespread adoption of qSOFA and Sepsis-3. The British approach using the NEWS score appears superior.
Addendum: After posting this, I received the following poster from Drs James Price and Narani Sivayoham of St. George’s University Hospital in London. Similar to Churpek et al., their group found a very low sensitivity for qSOFA:
Addendum 2: This is in response to a discussion with Jon-Emile (see below).
- Churpek MM et al. qSOFA, SIRS, and early warning scores for detecting clinical deterioration in infected patients outside the ICU. E-pub ahead of print in Am J Respir Crit Care Med.
- Blogs on Sepsis-3.0
- Top 10 problems with sepsis 3.0 (Pulmcrit)
- Sepsis 3.0 with Merv Singer, Additional thoughts with Cliff Deutschman (EMCrit)
- Sepsis 3.0 (Rebel EM)
- Sepsis 3.0 – No thank you (First 10 in EM)
- Sepsis definitions and diagnosis (LITFL)
- Sepsis: Redefined (FOAMcast)
- Batman, the sofa, and the latest sepsis definitions (St Emlyns)
- Sepsis isn't a disease (Intensive Care Network)
- Critique of Sepsis-III (Deranged Physiology)
- Debate in the literature
- UK Sepsis Trust toolkit & statement regarding Sepsis 3.0, also see interim statement about sepsis 3.0.
- New sepsis criteria: A change we should not make (American College of Chest Physicians)
- The misapplication of severity-of-illness scores towards clinical decision making. Moskowitz A et al. AJRCCM 2016.
- Change is not necessarily progress: Revision of the sepsis definition should be based on new scientific insights. Cortes-Puch et al. AJRCCM 2016
- It might have been more appropriate to designate the primary outcome as the combined outcome of mortality or ICU transfer. The real goal of identifying patients with sepsis is to transfer them to the ICU (and prevent mortality), therefore it would be logical to include ICU transfer in the primary outcome.