In the biomedical industry's relentless war against clinical reasoning, a multitude of biomarkers have been developed that promise assistance in the diagnosis and management of sepsis. In this ocean of mediocrity, procalcitonin has risen to the top. Since its predictive value is only slightly better than chance alone, it behooves those promoting its value to accentuate our limitations as medical providers, rather than emphasize its worth, questionable as it may be. Preying upon the clinical doubt that is inherent in the diagnosis and treatment of septic patients, the makers of procalcitonin choose to exploit these deeply rooted insecurities.
The first of these insecurities called into question is whether our clinical judgment as Emergency Physicians is sensitive enough to identify the subtle signs of early sepsis. The proponents of procalcitonin argue that by the time the disease becomes obvious enough for our clumsy clinical faculties to identify, it is too late. The thought is procalcitonin will be able to identify these patients earlier in their disease process, allowing us to intervene in advance of clinically obvious sepsis. Wacker et al recently published a systematic review and meta-analysis considering this very subject (1). Published in Lancet Infectious Disease in May 2013, this meta-analysis examined 31 data sets of ED and ICU patients to determine procalcitonin’s diagnostic capabilities. The pooled sensitivity and specificity of procalcitonin was 77% and 79% respectively, with an AUC of 0.85. The authors point out that a great deal of heterogeneity exists among the included studies owing to the multitude of cutoffs used in the individual trials, each one retrospectively selecting the threshold that provided the optimal performance of the assay in question. Given this, it is not unreasonable to conclude that when used clinically procalcitonin will perform worse than even these mediocre diagnostic characteristics suggest. In fact in an analysis of this very publication by Rucker et al, the authors examine what occurs to the accuracy of procalcitonin when you statistically account for the bias introduced by retrospectively selecting cutoffs that optimize the sum of procalcitonin’s sensitivity and specificity (2). Using a method known as the Youden Index (3) the authors calculate a more realistic sensitivity and specificity of procalcitonin , 72% and 73% respectively. Clearly not nearly accurate enough to be utilized clinically.
Notably, this meta-analysis addresses how well procalcitonin identifies sepsis in a vacuum. It does not address whether procalcitonin adds anything to our clinical ability to identify these patients. To answer this we have to examine procalcitonin's ability when compared to clinical judgment. In a study by Maisel et al published in European Journal of Heart Failure, the authors examined procalcitonin’s ability to differentiate bacterial infectious causes of dyspnea from non-infectious or viral causes (4). Using the gold standard of final hospital diagnosis made by two cardiologists and one pulmonologist blinded to the results of procalcitonin assay, the authors compared procalcitonin’s ability to identify pneumonia to Emergency Physicians’ unstructured judgment. Of the 1641 patients enrolled, 6.8% (112) had the final the diagnosis of pneumonia. The remainder had congestive heart failure, COPD, asthma, bronchitis, ACS, influenza, or various other maladies.
Similar to the trials included in the meta-analysis, the authors of the Maisel trial retrospectively fitted the diagnostic threshold of procalcitonin to optimize its diagnostic capabilities. In this trial it happened to be 0.25 ng/mL. Ideally this cutoff should be validated in a new prospective cohort to see if this threshold remains stable. Even with these bolstered numbers the test performed horribly. The AUC of procalcitonin for the diagnosis of pneumonia was 0.723, clinically useless. Especially given that the physician’s gestalt performed far better (AUC 0.84). When a decision tool was built using multivariate logistic regression combining clinical judgment and procalcitonin values, the AUC increased to 0.863. Though the authors claimed statistical significance, clinically this adds very little to physician’s judgment alone. Especially given the procalcitonin threshold was retrospectively fitted for optimal performance. The authors attempt to account for this by performing a bootstrap analysis in which the AUC of clinical judgment and clinical judgment plus procalcitonin were 0.834 and 0.857 respectively. Not only did clinical judgment outperform procalcitonin but so did chest x-ray findings. The authors report the ability of chest x-ray to diagnose pneumonia in this cohort with an AUC of 0.79. They go on to report the added benefit procalcitonin provides to chest x-ray (AUC of 0.864). They failed to report the combined diagnostic abilities of clinical judgment and chest x-ray, the classical way in which pneumonia is diagnosed. Despite the authors statistical chicanery they fail to provide a convincing argument for the utility of procalcitonin.
In an ironic juxtaposition, proponents of procalcitonin not only attack our failings in identifying patients who require antibiotic therapy but they further allege, on the rare occasion we do provide adequate antibiotic coverage we do so for longer than is required. In this vicious attack on our antibiotic stewardship, procalcitonin apologists testify that this biomarker's mediocre specificity, which is only equaled by its mundane sensitivity, can somehow guide the course of antibiotic use in sepsis. Despite these claims, multiple studies examining this question have failed to show a benefit. The most notable is the PRORATA trial, published in The Lancet in 2010 (5). In this trial, Bouadma et al compare 28 and 60-day mortality and antibiotic use in ICU patients when procalcitonin was used to guide therapy compared to unstructured judgment. The authors randomized 621 patients to either traditional management or management dictated by procalcitonin-based protocol. The authors found a statistically significant increase in number of days without antibiotics in the procalcitonin group without statistically increasing mortality. What the authors fail to highlight was though 28-day mortality was not statistically different (21.2% in the procalcitonin group vs 20.4% in the control group) overall, the procalcitonin group performed worse. 60-day mortality was 30% in the procalcitonin group compared to 26.1% in the control group. Relapse rate of 6.5% in the procalcitonin group compared to 5.1% in the controls. Incidence of superinfection was 34.5% in the procalcitonin group compared to 30.9% in the controls. Length of stay in the ICU was 15.9 days in the procalcitonin group compared to 14.4 in the controls. Though procalcitonin may reduce the utilization of antibiotics, it does not appear this reduction is of any clinical benefit.
In the PASS trial, published in Critical Care Medicine in 2011, Jensen et al examined a similar procalcitonin-based protocol to guide antibiotic stewardship (6). Like the PRORATA trial, the authors randomized 1200 ICU patients with suspected infections to either standard care or a procalcitonin-based protocol. In this cohort the authors were not able to identify a decrease in antibiotic use in the procalcitonin group, in fact a procalcitonin-based protocol seemed to increase antibiotic use in this Danish population. Though the mortality rate was similar in the two groups ( 31.6% vs 32%), patients in the procalcitonin group spent more days on the ventilator, more days in the ICU and more days with organ failure.
Clearly we are not as bad in the diagnosis and management of sepsis as the makers of procalcitonin assays would have you believe. Whatever deficits we do possess have not been corrected with the addition of procalcitonin. Brain Natriuretic Peptide (BNP) forced its way into clinical use in a similar fashion to which procalcitonin is now attempting. As a marker of a disease process with moderate prognostic capabilities, BNP when examined in a vacuum, using retrospectively determined thresholds performs adequately, but when tested in the clinical arena, does nothing to augment medical decision-making(7). Sepsis is a bad disease with unfortunate outcomes. These outcomes should not be viewed as a comment on our clinical abilities, but rather a statement of the severity of the disease itself. Not as our inadequacies but rather medical realities. Turning to an expensive biomarker incapable of providing the certainty desired is obviously not the answer. The schoolyard bully tactics of those promoting procalcitonin should not persuade us otherwise.
This post is dedicated to Dr. Eric Wasserman
Sources Cited:
1. Wacker et al. Procalcitonin As a Diagnostic Marker For Sepsis: a Systematic Review and Meta-Analysis. Lancet Infect Dis. 2013 May;13(5):426-35
2. Rucker et al. Procalcitonin As a Marker For Sepsis. The Lancet Infectious Dis. Volume 13, Issue 12, Pages 1012 – 1013, December 2013
3. Rücker G, Schumacher M. Summary ROC curve based on the weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat Med 2010; 29: 3069-3078
4. Maisel et al. Use of procalcitonin for the diagnosis of pneumonia in patients presenting with a chief complaint of dyspnoea: results from the BACH (Biomarkers in Acute Heart Failure) trial. Eur J Heart Fail. Mar 2012; 14(3): 278–286.
5. Bouadma et al. Use of Procalcitonin to Reduce Patients’ Exposure to Antibiotics in Intensive Care Units: a Multicenter Randomized Controlled Trial. The Lancet Vol 375. Feb 6th 2010
6. Jensen et al. Procalcitonin-guided interventions against infections to increase early appropriate antibiotics and improve survival in the intensive care unit: a randomized trial. Crit Care Med. 2011 Sep;39(9):2048-58.
7. Hohl CM. Should natriuretic peptide testing be incorporated into emergency medicine practice? CJEM. 2006 Jul;8(4):259-61.
- EM Nerd-The Case of the Partial Cohort - May 24, 2020
- EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
- EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019
Love it! Though it’s not commonly seen at my shop yet, I’ve started hearing the hospitalists muttering things about procalcitonin on an increasing basis. It’s only a matter of time before it takes hold, but I’m saving this as a talking-point reference for when the topic comes up. Thanks Dr. Spiegel, I know how much time and effort must go into creating a well-written and thoroughly-researched post like this.
Thanks Vince! This post was actually born out of very similar circumstances to what you describe. The reps are currently pushing procalcitonin for our shop. This was my argument against acquiring the assay. Only time will tell if it was anything but yelling into the wind…
Thanks again for your support!!
Rory, Wow… incredibly well written and thought out. Keep up the good work. I can’t friggin believe you are a second year resident. My guess is you had a previous life. Regarding procalcitonin… you make excellent points regarding the studies that have derived the optimal cut off for sensitivity/specificity & AUC. In addition, the test is studied in a heterogeneous population with patients who are at the extreme ends of disease process. So obviously the test should perform well in these patients. But, I am not interested in using a test when it is clinically obvious someone is sick or… Read more »
Hi Brian,
Thanks you so much for your kind praise. I agree completely, the subtle cases where you would want the test to provide guidance is probably where it is least useful. I’m impressed you have gone 8 years without ordering a BNP. I unfortunately cannot make that claim. I can say that I order only upon attending request and it has not once helped me manage a dyspneic patient.
Thanks again for the support!!
Dear EM Nerd, This is a pretty “entertaining” comment, but you are not doing either biomarker any justice. To bash PCT, you picked some papers to make your point, jumping to personal conclusions which are, forgive me, often wrong. Furthermore, you ignore many well designed other studies and reviews (including a recent Cochrane Analyses),. Also you have a slight (forgive me again) USA arrogance in dealing with this topic. Many European ER and ICU docs would strongly disagree with your conclusions, which is throwing out the baby with the bath. This forum may not be the right place for a… Read more »
Thank you so much for your thoughtful comments. While I understand your feelings of protectiveness towards a biomarker you have nursed from infancy, I must respectively disagree regarding procalcitonin’s clinical utility. Though I did not include every study examining procalcitonin’s abilities to predict bacterial infections or guide antibiotic therapy I have read the majority of them. I chose not to mention all of them because they demonstrated very similar test characteristics and add little additional evidence other than confirming procalcitonin’s mediocrity. I think the meta-analysis by Wacker et al demonstrates this nicely. As far as the diagnostic threshold utilized in… Read more »
Dear EM Nerd, I can see your point. The publication field of PCT is pretty messy. If you have no personal experience, and all your inside comes from reading and disecting papers (as you are very good at here in your blog), then it is indeed difficult to trust this marker. One problem is the cut-off. This is something you cannot apply without using your clinical judgement. In your example above, they used a very low cut-off (0.1 ng/ml). To use this for antibiotic stewardship is questionable in the first place. The “one cut-off fits all patients and all clinical… Read more »