We are constantly hunting for newer and better sources of information.  This leads to a perpetual generation of newer lab tests, different hemodynamic gizmos, and fresh decision tools.  Some of these pan out.  Most don’t.

Why are these new sources of information generally disappointing?  The answer is simple:  we’re already doing pretty well.  In order for a new source of information to help us, it needs to represent a substantive improvement over what we already have.  We don’t need more information (that’s just noise) –  we need better information. 

This principle was illustrated by the above article comparing various decision instruments versus clinician judgement.^​1​  Decision instruments generally perform well as a stand-alone tests.  However, when compared to the judgement of treating physicians, decision instruments seldom add anything.  This shouldn’t be surprising.  Decision tools are terrific at processing a few bits of information, but they ignore lots of other information.

background:  the Rothman Index (RI)

The electronic medical record contains a massive and ever-increasing amount of information.  This naturally leads to the question of whether decision instruments could be embedded into the electronic medical record.  Personally, I’ve been expecting that EPIC would gain awareness for a while now.  My hope is that self-aware EPIC would stop bugging me with dumb alerts and perhaps help with potassium repletion, but I’d settle for a self-aware EPIC which alerts me about deteriorating patients.

The Rothman Index is an illness-severity index embedded within the electronic medical record.  It continuously tracks 26 variables shown above.^​2​  This data is fed into a proprietary algorithm, yielding a numerical score which mirrors how sick the patient is.  Deterioration is predicted by rapid drops in the Rothman Index, or by a low absolute value of the Rothman Index. 

The Rothman Index has been promoted for a variety of uses, for example:

• Identifying sick patients who need more urgent attention (e.g. transfer to ICU).
• Identifying the healthiest patients in the ICU, who may be ready to transition to the floor.
• Identifying patients who aren’t improving, who may benefit from palliation.

The Rothman Index is a proprietary algorithm, which is the sole product marketed by the PeraHealth company.  This is currently a multi-million dollar enterprise which is growing rapidly. 

To really understand this, we need to lift the hood and explore how the Rothman Index is obtained^​3​

1. The degree of abnormality in each of the 26 variables is calculated as shown above (in a normalized fashion, such that results are roughly comparable across variables).
2. The Rothman Index is calculated as the sum of the abnormality in each variable:

For comparison, this is exactly the same way that most early warning scores are designed (e.g. the NEWS score in the figure below). It's a solid design, but it's nothing revolutionary.

Thus, the Rothman Index is a fairly blunt instrument. There's no higher-level assessment of the complex interaction between different variables. There's no sophisticated trending of variables over time. There's no artificial intelligence or machine learning going on. This is basically a souped-up early-warning system.

Compared to simpler early warning systems (such as NEWS), the Rothman Index has a lot more data inputs. This might seem great, but not necessarily: more isn't always better. The problem is that data points hold equal weight (they are summed together). Therefore, random flux in one variable (say, risk of pressure ulceration) could cancel out changes in another variable (say, new-onset bradycardia). Additional data inputs could merely add noise, obscuring signals from the most important variables.

More specifically, in contrast to most early warning systems, the Rothman Index incorporates additional data from labs and nursing assessments. Labs and nursing assessments aren't performed very frequently. Thus, the Rothman Index might be better positioned to track a patient's trajectory over a period of days-months, but it wouldn't be expected to work better over a period of hours. Indeed, the Rothman Index might actually be worse at the immediate detection of acute physiologic derangements, when compared to traditional early warning systems (since most of the Rothman Index parameters cannot respond rapidly to acute deterioration).

recent literature on the Rothman Index

Winter MC et al 2019:  Beyond reporting early warning score sensitivity:  The temporal relationship and clinical relevance of “true positive” alerts that precede clinical deterioration.

Most studies relate the RI to subsequent clinical deterioration.  This ignores the fact that clinicians may already be fully aware that the patient is deteriorating (without needing the RI to help them).  Indeed, since the RI is based on nursing assessments, it’s likely that in some cases an increased RI may reflect that the bedside nurse has already realized the patient is getting worse. 

This is a retrospective study of the pediatric Rothman Index performed at the Children’s Hospital of Philadelphia (CHOP).^​4​  Rothman Index alerts were identified which preceded clinical deterioration (“true positive” alerts).  Medical records were carefully reviewed to determine whether the RI alert occurred before or after clinicians realized the patient was deteriorating. 

In 90% of cases, clinicians had already recognized that the patient was deteriorating, before the RI alert.   Thus, the vast majority of true-positive RI alarms didn’t affect management at all.

This study is limited due to its performance at a prominent medical center, which has more resources than most.  Thus, it’s possible that at a different center with less robust nursing staffing, clinicians would recognize deterioration later (rendering the the RI relatively more useful). 

Majeed J et al 2019: Utility of the Rothman Index in cancer patients who have in-hospital cardiac arrest.

This is a retrospective study of 59 patients who suffered an in-hospital cardiac arrest at Memorial Sloan Kettering Cancer Center.^​5​ On the day of cardiac arrest, only 29% of patients had a positive Rothman alert. This indicates that in actual practice, the sensitivity of the Rothman Index may be poor.

Fitzpatrick N et al 2018:  Impact of the Rothman index on delay of ICU transfer for hematology and oncology patients deteriorating in wards.

This is a before/after trial at Penn State Medical Center, comparing patients transferred to the ICU before and after implementation of the Rothman Index.^​6​  For patients in the “before” group, archival records were evaluated to retroactively calculate what the measured Rothman Index would have been for those patients. 

The data in both groups of patients is eerily similar.  Perhaps most notable is the average delay from low RI scores to ICU transfer – which was the same in both groups.  This suggests that knowing the RI didn’t actually accelerate care. 

Gotur DB et al. 2018:  Analysis of Rothman Index data to predict post-discharge adverse events in a medical intensive care unit. 

This was a single-center retrospective study which compared RI at time of ICU discharge with the risk of an adverse event within the next 72 hours (e.g. ICU re-admission, rapid response activation, or cardiopulmonary arrest).^​2​

Patients with lower RI scores did have higher risk of adverse events, as shown above.  However, most patients with low discharge RI scores (in the Red Zone) didn’t have an adverse event.  Alternatively, some patients with the highest RI scores (in the Blue zone) did have an adverse event.  And most patients with a RI “warning” prior to transfer didn’t have an adverse event.

The authors found that the best cutoff value was a RI score of 50.  Patients with a discharge RI above 50 had lower odds of an adverse event (Odds Ratio 0.29). 

This performance isn’t particularly impressive.  An Odds Ratio of 0.3 represents a weak/moderate strength of evidence, which isn’t reliable enough to drive decisions regarding ICU discharge.  Furthermore, this cutoff hasn’t been validated in any subsequent cohorts of patients. 

Alarhayem AQ et al.  2019:  Application of electronic medical record-derived analytics in critical care:  Rothman Index predicts mortality and readmissions in surgical intensive care unit patients. 

This is another single-center retrospective study which compared RI at the time of ICU discharge with adverse events (readmission within 48 hours).^​7​  RI was related to the risk of readmission as follows:

Once again, there is a relationship between RI and readmission – but it’s not robust.  The vast majority of these patients did fine, regardless of their RI.  Changes in RI prior to transfer were also not a reliable predictor of ICU readmission.

Arnold J et al.  2019:  Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalized general internal medicine patients.

This is perhaps the best study of the RI, because it involves a prospective comparison of the RI against clinician judgement.^​8​ 

The study was performed on the internal medicine wards at the University of Pittsburgh.  At the end of every day, the intern on the medicine team was asked to predict the risk that each patient on the team would deteriorate overnight.  The validity of this prediction was compared with the patient’s Rothman Index at that time.  During the study period the RI wasn’t integrated into clinical workflow and had not been advertised to the clinical staff, so RI didn’t influence the interns’ predictions.

The RI and the interns’ predictions about who would deteriorate had the same accuracy!  This should help debunk the concept that the Rothman Index has magical properties which must be heeded, regardless of clinician judgement.  In this study, the Rothman Index is performing at the level of an intern.

Rothman Index success stories

Proponents of the Rothman Index will point to before-after studies performed at Yale and Houston Methodist Hospital.   For the sake of clarity and fairness, the entire study from Yale is reproduced above.  These two studies share several similarities:

• They were published only in abstract form in the British Medical Journal of Quality and Safety.  As such, the methodology is very scant. (They aren't listed in PubMed, but can be found here and here.)
• The Rothman Index was introduced as one component of an aggressive rapid response team.  For example, at Yale a “nursing SWAT team” rounded pre-emptively on the sickest patients.  At Methodist, pro-active rounding on sicker patients was done by nurse practitioners. 

It appears that these strategies improved patient care.  However, this doesn’t clarify which component of these strategies was actually helping the patients – was it the Rothman Index, or was it the SWAT nurses?  My guess is that a more aggressive nursing culture with pre-emptive management of the sickest patients made the difference (not the Rothman Index). 

evidence summary:  where does this leave us?

According to the main page on the PeraHealth Website:

The Rothman Index detects sepsis earlier, lowers cost per case, prevents unplanned ICU transfers, reduces 30-day readmissions, and optimizes palliative care consults. 

–https://www.perahealth.com/ accessed 1/24/20

These are extraordinary claims, which aren’t well supported by available evidence. Related early warning systems have variable performance, so within a larger context these claims are far-fetched.^​9​ A more realistic description of the Rothman Index might be as follows:

• Taken in isolation, the RI performs surprisingly well.  It correlates strongly with mortality and a variety of outcomes. 
• When added to usual clinical practice, the RI doesn’t provide novel information or improve patient care. 
• The combination of the RI plus aggressive nursing interventions does improve outcomes.  This probably reflects the benefits of high-level nursing care, rather than the RI. 

This should not be surprising.  This is the way most decision-support aids work.^​1​  On paper they look fabulous, but in practice they don’t tell us much that we didn’t already know. 

our system has problems, but Rothman Index isn’t the cure

The Rothman Index does highlight one problem in modern American healthcare:

• Nurses spend lots of time documenting detailed organ-system assessments of their patients.
• These assessments are valid, but nobody pays attention to them (they’re often buried deep within the electronic medical record, in places where physicians never look). 
• One strength of the Rothman Index is that it shines some light on these assessments, allowing them to have some utility.

Focusing more on the Rothman Index would double down on the importance of nurses' entering assessments into the electronic medical record.  This could require nurses to spend even more time in front of computers, and less time with their patients.  Likewise, physicians would be focused on tracking the Rothman Index – rather than talking with bedside nurses.  Everyone would be sucked further into the electronic medical record. 

There is an alternative, and perhaps better solution:

• Don’t use the Rothman Index.
• Stop forcing nurses to spend half their time documenting in the electronic medical record.  Let nurses be nurses, not data entry technicians. 
• Work on improving communication between nurses, physicians, and rapid response teams.  Nurses should be empowered to activate a rapid response team or call for help based on their best judgement – without needing the Rothman Index to back them up. 
• Pre-emptively rounding on the sickest patients in the ward is a terrific idea but this, again, can be done perfectly well without a Rothman Index.  You don’t need the Rothman Index to know which patients are the sickest –  just ask the closest intern!

• The Rothman Index is an early warning system which is integrated into the electronic medical record. It is similar to prior paper-based early warning systems designed to detect instability (e.g. NEWS), albeit with the addition of information from labs and nursing assessments.
• The Rothman Index works well on paper as a predictive tool.  However, recent evidence suggests that the Rothman Index fails to add true value in practice, for several reasons:
  • The Rothman Index often provides clinicians with information which is redundant to what they already know (e.g. alerts commonly occur after the treating team has already realized the patient is deteriorating).
  • The Rothman Index has limited predictive ability.  In one study, the Rothman Index was no better than an intern at predicting patient deterioration. In another, only 29% of patients with cardiac arrest had a RI alert that day.
• The Rothman Index does fill a void in the healthcare system, which is caused by lack of physician attention to nursing assessments.  This is a problem, but there are better ways to fix it.  The answer is probably for all of us to spend less time looking at computers – and more time talking with each other.

going further

• Maryland CCC Project: Lecture by Dr. Rothman on the Rothman Index.

• Author
• Recent Posts

Social Me

Josh Farkas

Josh is the creator of PulmCrit.org. He is an associate professor of Pulmonary and Critical Care Medicine at the University of Vermont.

Social Me

Latest posts by Josh Farkas (see all)

• PulmCrit Wee – Loading infusion auto-titration (LIAT) for infused medications with intermediate half-lives - March 23, 2025
• PulmCrit Wee: Michelin Chest Syndrome - March 15, 2025
• PulmCrit: ADAPT and SCREEN trials are full of sound and fury, signifying little - December 13, 2024