February 7th 2019 marked a auspicious day in the history of the NEJM and its relationship with the pharmaceutical industry. In 2015, the NEJM shifted their significant weight in support of a strong union between industry and medicine¹,²,³. With the simultaneous publication of four pharmaceutical sponsored trials, each brimming with its own unique form of methodological and statistical tomfoolery, The Journal has made it exquisitely clear this partnership is as cohesive as ever.

The first of these studies, by Stets et al, examined the use of omadacycline, a novel aminomethylcycline antibiotic, for the treatment of community acquired pneumonia (CAP)⁴. Published alongside an almost identical article examining omadacycline use in soft-tissue infections⁵, these industry sponsored double-blind, double dummy, randomized non-inferiority trials, purport this new expensive antibiotic as good as the traditional and far more affordable approach.

Given the following excerpt,

“Paratek Pharmaceuticals designed and conducted the trial and prepared the statistical analysis plan. Analyses were performed and data interpreted by Paratek Pharmaceuticals in conjunction with the authors.”

It is fairly evident that this article was not intended to further scientific progress, but rather to sell a shiny new product. The authors enrolled adult patients greater than 18 years of age with radiologically confirmed pneumonia and a pneumonia severity index (PSI) risk class II or greater. Using a fairly complex non-inferiority analysis, the authors report omadacycline to be non-inferior to moxifloxacin. And while by all accounts this novel antimicrobial appears to be non-inferior and likely equivalent to moxiflaxacin, it is the intention and the methodology which we should call into question.

If the authors intended to demonstrate the clinical benefits of omadacycline, why did they employ a non-inferiority trial design? Non-inferiority trials are utilized to investigate a treatment strategy that, while not likely to be superior to our traditional approach, offers some logistical or financial advantages. In such cases a non-inferiority trial asks, given the advantages offered by the use of this novel approach, is this new treatment no worse than standard care? But what logistical or financial advantages does omadacycline offer over standard care? Is it financially more viable? Not likely, a 7-day course of omadacycline has been estimated to cost over $3,000⁶. Far more than the cost ($30-$108) for a course of moxifloxacin. Does it offer a logistical advantage over our other antibiotic regimes? Omadacycline is a once a day antibiotic, but so is moxifloxacin and levofloxacin. A non-inferiority trial design was utilized not because omadacycline offers legitimate benefits outside its clinical efficacy, but rather because the authors wanted to give the appearance of a positive study.

A non-inferiority trial offers a number of advantages to manufacture a positive trial. The most obvious is one does not have to demonstrate superiority of the novel treatment, but rather simply demonstrate that it is not worse than standard care. Because of this, blinding becomes a far less efficacious method of controlling bias, as assessors do not have to know what treatment group a patient is assigned, to introduce bias into the trials results. Simply assigning similar outcomes for all participants, independent of group assignment, will give the appearance of non-inferiority. Thus a trial can claim a methodologically rigorous defense against bias (double-blind, double dummy, randomized), while simultaneously undermining the efficacy of such measures. The non-inferior trial design offers the additional benefit of permitting authors to define non-inferiority. In this case, Stets et al stated that to be considered non-inferior, omadacycline could demonstrate no more than an 10% increase in treatment failure when compared to moxifloxacin. The authors cited data suggesting patients with CAP, experience a 20% ARR with standard antibiotic therapy when compared to placebo.They justified ther huge margin of inferiority by stating it is half of the treatment effect when standard therapy is compared to placebo. But these estimates of antibiotic efficacy are based on trials conducted in the 1940s and 1950s⁷,⁸,⁹, during which the use of antibiotics resulted in a massive improvement in mortality. Despite these extraordinary benefits, mortality in the patients who received antibiotics still ranged from 5-17%. Mortality in the current trial was 2.1% and 1.0% in the omadacycline and moxiflaxacin groups, respectively. Clearly this modern-day cohort of patients with CAP is not the same as the historical comparisons, and the authors primary outcome, early clinical response, is not equivalent to death. Given this, a 10% absolute decrease in efficacy for a drug that offers little or no logistical or financial advantages over standard care is intended only to ensure the results of the trial will be positive.

In the face of growing antibiotic resistance, new agents are urgently needed. And while omadacycline does have theoretical activity against MDR organisms, that was not evaluated in either of the two trials in the NEJM⁴,⁵.

The next trial published was the PIONEER-HF trial, by Velazquez et al¹⁰. The authors examined the use of angiotensin–neprilysin inhibition in patients admitted to the hospital with acute decompensated heart failure. The authors enrolled 881 patients, 18 years of age or older, with the primary diagnosis of acute decompensated heart failure to receive sacubitril–valsartan (Entresto) or enalapril. The authors claim success, citing a positive p-value for the primary outcome in favor of the sacubitril–valsartan group.

Like the previous trials, the makers of Entresto had direct control of the trial,

“Novartis was the sole sponsor and conducted the trial in collaboration with the Duke Clinical Research Institute (DCRI) and the Thrombolysis in Myocardial Infarction (TIMI) Study Group.”

Rather than use a non-inferiority trial design and a generous non-inferiority margin, the authors ensured success by the use of a surrogate outcome, time-averaged proportional change in the NT-proBNP levels. A surrogate endpoint is a marker of disease that trialists use as a substitute for a patient oriented outcome (mortality, symptom burden, etc), usually because of ease of measure, with the assumption that the surrogate marker translates into a meaningful change in clinical outcomes. This is often not the case. One of the most famous examples of this is the CAST trial¹¹, which examined the use of ventricular arrhythmia suppression in patients with a myocardial infarction (MI). Following an MI, it was observed patients who experienced more frequent ventricular ectopy had higher mortalities than those without such arrhythmias. Medications that reduced this ectopy were considered beneficial as a reduction in ventricular ectopy was considered a surrogate for reduction in mortality. Epstein et al randomized patients, following an MI, to arrhythmia suppression using encainide, flecainide, moricizine or placebo. And while these medications decreased the frequency of ventricular ectopy, this did not translate into a reduction in mortality, in fact the anti-arrhythmic group had a higher 1-year mortality.

In the case of the PIONEER-HF trial, the authors found no difference in the rate of death, rehospitalization for heart failure, left ventricular assist device implantation, listing for heart transplantation, or increase in medical therapy for heart failure. There was no difference in any clinically important metric measured between the two groups. And yet, the trial was published in the NEJM based off a statistically positive surrogate endpoint of no clinical consequence.

The final trial published was the ANNEXA-4 trial¹². Like the previous trials,

“The Population Health Research Institute (PHRI) at McMaster University and the industry sponsor, Portola Pharmaceuticals, jointly designed the study, and both selected sites and supervised monitoring.”

Only on this occasion, Connolly et al abandoned the use of the RCT all together. In its place conducted a prospective, open-label, single-group study to promote the efficacy of andexanet alfa as an antidote to the anticoagulation effects of factor Xa inhibitors. The authors enrolled adult patients on an oral Xa inhibitor who presented with an episode of acute major bleeding. Patients received a 30-minute bolus followed by a 2-hour infusion of andexanet alfa. Anti–factor Xa activity was measured before, during and after treatment. Enrolling 352 patients, the authors report a marked reduction in anti–factor Xa activity during infusion, as well as 4, 8, and 12 hours after andexanet infusion.

Like the previous trial, Connolly et al employed a surrogate outcome as a marker of andexanet’s efficacy. Are we to assume a decrease in anti-factor Xa inhibition levels translates to a decrease in Xa inhibition, which in turn translates to an increase in hemostasis? The clinical reality supporting this required leap of faith is unclear. The lack of a control group allows the authors to boast about how frequently hemostatic efficacy was achieved without a comparator to keep them honest. The authors report excellent or good hemostatic efficacy occurred in 81% of the patients. The 18.1% judged to have poor hemostatic efficacy were relegated to the supplementary appendix. 10% of the patients experienced one or more ischemic events, 61.8% of these came in the form of MIs or ischemic strokes.

Without a control group it is impossible to differentiate the hemostatic efficacy of andexanet alfa from the general improvement of patients’ coagulation capabilities when the offending agent is removed. The authors excluded patients in whom surgery was planned within 12-hours of andexanet alfa treatment, patients with intracranial hemorrhage and a GCS less than 7, or an estimated hematoma volume of more than 60 cc. Patient selection alone could account for the observed results. In fact, there was no significant relationship between hemostatic efficacy and a reduction in anti–factor Xa activity during andexanet treatment. Without a control group with which to compare we are unable to truly assess andexanet alfa’s hemostatic abilities. Simply by selecting a cohort of patients that is likely to do well no matter what reversal agent administered, the authors ensured the appearance of andexanet alfa’s efficacy.

While each of these articles offer their own distinct set of insults against empiricism, they do so in close temporal and spatial proximity, all published in the NEJM on the same day. The methods deployed in these articles to subvert the scientific method are not unique to the NEJM. They are pervasive throughout the scientific literature and represent the dire crisis which faces evidence-based medicine today, a systematic assault, intent on distorting and manipulating scientific inquiry for its own financial gain.

Sources Cited:

• Author
• Recent Posts

Rory Spiegel

EM and Critical Care Attending
University of Georgetown
Resuscitation and Critical Care Fellowship Graduate

Creator
EMNerd.com

Latest posts by Rory Spiegel (see all)

• EM Nerd-The Case of the Partial Cohort - May 24, 2020
• EM Nerd: The Case of the Sour Remedy Continues - January 20, 2020
• EM Nerd-The Case of the Adjacent Contradictions - December 23, 2019