A peasant traveling home at dusk sees a bright light traveling along ahead of him. Looking closer, he sees that the light is a lantern held by a ‘dusky little figure', which he follows for several miles. All of a sudden he finds himself standing on the edge of a vast chasm with a roaring torrent of water rushing below him. At that precise moment the lantern-carrier leaps across the gap, lifts the light high over its head, lets out a malicious laugh and blows out the light, leaving the poor peasant a long way from home, standing in pitch darkness at the edge of a precipice.
-Welsh tale describing Will-o-the-Wisp
So much of what we do in Emergency Medicine is translating shades of grey into dichotomous patient oriented decisions. Truth in medicine is a fluid, tenuous state, very rarely encountered in the chaos of the Emergency Department. More often than not we are forced to act in varying states of uncertainty. Naturally we search out specific data points in this fog of ambiguity that we believe will provide guidance through the unknown. And yet, some of these beacons are just as likely to lead us astray as they are to provide safe passage.
One such variable is a history of loss of consciousness (LOC) in a patient suffering from a minor head trauma. Despite a multitude of contradictory data, LOC has persisted in the mind of the practitioner (often times in isolation) as a relevant branch-point in deciding who does and does not require further downstream investigations (2). The most recent excavation of the PECARN dataset, published in JAMA Pediatrics, should serve to remind us that just because a variable is found to have a statistical association to the endpoint in question, this does not necessarily mean it is a useful factor to guide clinical decision- making (2).
In this latest dive into the PECARN dataset, Lee et al set out to examine how influential LOC was in predicting clinically significant traumatic brain injury (ciTBI). In the original derivation and validation cohort, by Kupperman et al, LOC was identified as one of the six variables with a strong enough predictive value to be included in the formal decision rule (1). The original PECARN data set was a mammoth undertaking, which prospectively evaluated 42,412 pediatric patients presenting to the Emergency Department after experiencing a minor head injury. Of this group only 780 patients (1.8%) were found to have any evidence of TBI on CT. Only 376 (0.9%) of these patients had injuries of clinical relevance, of which only 60 patients (0.14%) required any form of neurosurgical intervention. Given this extremely low rate of ciTBI, one could argue that the PECARN authors had already identified a cohort of patients at incredibly low risk for relevant injury and any further risk stratification would be futile. Despite this the original authors derived and internally validated two age specific (< 2 years old and> 2 years old) decision rules that boasted negative predictive values of 100% and 99.95% respectively. This data set remains the most robust clinical decision rule derived to date in the pediatric population despite lacking sufficient external validation, incomplete follow up (1/5 of the 64.7% of the patients who did not undergo definitive testing were lost to follow-up), and the fact that the rule was outperformed by physician’s unstructured judgment (1).
Lee et al sought to improve, at least conceptually, on the diagnostic characteristics of the PECARN decisions rules by addressing the added value isolated LOC provides in identifying patients with ciTBI. The authors defined isolated LOC in two specific fashions. In one, termed PECARN-isolated LOC, they identified patients who experienced LOC without any of the other factors that make up the PECARN decision rules. The second utilized the expanded definition of LOC, which included predictors from other commonly used decision rules for head injury (Nexus 2, the New Orleans criteria, and the Canadian head CT rule). It is important to note that the expanded definition of LOC did not include mechanism of injury as a relevant predictor of ciTBI (2).
Of the 42,412 patients, 6,286 (15.4%) were found to have suspected or confirmed LOC. An interesting side note was that out of the 6,286 patients with LOC, 5,010 had a head CT performed, the majority of which the treating physician recorded the history of LOC as being the primary reason for the scan (demonstrating that even in this cohort LOC was considered a clinically important factor for predicting injury). Of the patients with a history of LOC, PECARN-isolated LOC was present in 2,780 (47.5%) patients. In the subgroup of patients with PECARN-isolated LOC, the incidence of TBI on CT was 1.9% and the incidence of ciTBI was 0.5%. Unfortunately the expanded definition of isolated LOC was far less useful as only 576 (9.4%) of patients with LOC met its’ criteria, most likely do to the inclusion of “any traumatic scalp findings” as a relevant predictor. Of those that did meet these impossible standards, only 0.9% were found to have TBI on CT and 0.2% of these patients had a clinically relevant injury. In the PECARN cohort if LOC was used independently as a decision point for head CT, the sensitivity and specificity of identifying ciTBI would be 49.5% and 85.4% respectively. Clearly not the beacon of light we presume.
What is important to remember is that a statistically significant odds ratio found by using a multifactorial regression model does not directly translate into a clinically useful predictor. Multifactorial regression in all its forms is a statistical attempt to isolate the effect of one variable’s ability to predict the outcome in question. Essentially it is the graphical illustration (the slope of the line indicating the strength of the association) of how one variable affects another while a mathematical attempt is made to control for other factors (3). Despite its statistical authority, finding an independent association between a variable and the outcome in question is not the same as studying a group of patients otherwise well with the exception of the variable in question (LOC for example). Moreover the odds ratio that is typically reported as the result of a multifactorial regression model does not intuitively explain the clinical relevance of this correlation (3).
The utility of isolated LOC for predicting clinically significant TBI seems to have undergone this very mathematical augmentation. Although LOC has consistently demonstrated a statistically independent association with ciTBI, when applied clinically in patients with isolated LOC its predictive value is minimal. During the derivation cohort of the Canadian head CT rule Steill et al found LOC was independently associated with ciTBI (4). However when used clinically they found only 0.4% of patients with LOC had a clinically relevant ciTBI requiring intervention, and most of these could be identified simply by assessing the patient’s mental status in the ED (5). In the NEXUS 2 cohort, LOC was identified as a predictor of ciTBI but failed to maintain clinical relevance when assessed using a multifactorial model (6). Additionally if LOC was used to decide which patients in this cohort would receive further imaging it would have resulted in a sensitivity and specificity of 48% and 63% respectively (6). In the original PECARN cohort the predictors that identified the bulk of the patients with ciTBI were altered mental status (AMS) or clinically obvious signs of skull fractures. These factors alone identified the bulk of patients with ciTBI. If patients did not present altered or with obvious signs of skull fracture, then their risk of ciTBI was incredibly low (0.9% in the under 2 years old group and 0.8% in the over 2 years old group). The remainder of the predictors found in the PECARN decisions rules, including LOC, did very little to further risk stratify patients (1).
What this can be reduced down to is our fear of the clinically occult head bleed. Based on the idea that the skull is a lead box blocking the transmission of potential chaos within from our external eye until it’s too late to intervene. This fear is driven by anecdote passed down from attending to resident in some form of modern-day oral history. Clearly these stories are not supported by the literature and the reality is these cases of clinically occult intracranial bleeding are rare and often identifiable by high-risk features (elderly, anticoagulant use, etc). A history of LOC in an otherwise well-appearing patient provides us with little guidance in identifying these rare cases. Moreover the lack of LOC does not safely eliminate the risk of significant injury. Often times its absence will give us a false sense of security and like the solitary peasant, lead us far from home, standing in pitch darkness on the edge of a cavernous precipice…
Sources Cited:
- Kuppermann N, Holmes JF, Dayan PS, et al; Pediatric Emergency Care Applied Research Network (PECARN). Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374(9696):1160-1170.
- Lee LK, Monroe D, Bachman MC, et al. Isolated Loss of Consciousness in Children With Minor Blunt Head Trauma. JAMA Pediatr. Published online July 07, 2014. doi:10.1001/jamapediatrics.2014.361.
- Barrett, Tyler W. et al. Is the Golden Hour Tarnished? Registries and Multivariable Regression. Annals of Emergency Medicine , Volume 56 , Issue 2 , 188 – 200
- Stiell IG, Wells GA, Vandemheen K. et al. The Canadian CT Head Rule for patients with minor head injury. Lancet. 2001;357:1391-1396
- Stiell IG, Clement CM, Rowe BH, et al. Comparison of the Canadian CT Head Rule and the New Orleans Criteria in Patients With Minor Head Injury. JAMA. 2005;294(12):1511-1518. doi:10.1001/jama.294.12.1511
- Mower WR, Hoffman JR, Herbert M, et al, Developing a Decision Instrument to Guide Computed Tomographic Imaging of Blunt Head Injury Patients. J Trauma. 2005 Oct;59(4):954-9. (Nexus II)
- EM Nerd-The Case of the Partial Cohort - May 24, 2020
- EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
- EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019