Grein et al. just published a case series of patients with COVID-19 treated with remdesivir via a compassionate use program. I’ve been expecting this paper with cold dread for some time now. There are several reasons that this publication is a media show, rather than a serious scientific endeavor.
#1. Lack of a control group
As a multi-billion-dollar pharma giant, Gilead has the ability to perform RCTs. Administration of a relatively new drug with unclear side-effects should occur within the context of a controlled trial. Administration of unproven medications under the auspice of compassionate use opens the door to all sorts of unknown toxicities and dangerous practices.
Lack of any control group makes it extremely important that the patients were selected in an unbiased fashion, representing an average group of patients with COVID-19. Unfortunately, this wasn’t the case…
#2. Cherry-picking Patients
Remdesivir was aggressively sought-after by thousands of patients with COVID-19. Of these patients, 61 ended up receiving the drug. Why did these 61 patients receive medication, out of scores of patients applying to receive it?
The manuscript is extremely coy regarding how many patients applied to use remdesivir and how this application process worked. For example, there is no documentation of how many patients applied. There is no data comparing characteristics of patients included in the study versus patients denied medication. Given lack of any control group (#1 above), lack of transparency about this selection process is stunning.
The fact is that Gilead excluded many patients at a higher likelihood of dying (e.g., patients on vasopressors or patients with renal failure). The manuscript doesn’t mention vasopressors, but this was an exclusion factor in the United States.
Through reliable sources, I am aware of one case where a patient was initially denied remdesivir because the patient was too ill (renal failure, vasopressor requirement). Subsequently this patient improved clinically, so Gilead changed their mind and decided that the patient was a candidate for remdesivir! Thus, it seems that the selection process may have been designed to capture patients maximally likely to recover, rather than patients maximally likely to benefit from remdesivir.
#3. Loss of patients due to “no post Day 1 clinical data”
Of 61 patients initially included in the trial, 7 were excluded due to an absence of clinical data after the first trial day. This is disconcerting. What happened to these patients? Did they die from anaphylaxis? Did they get well, sign out against medical advice, and go party? This is unknown – but I’m worried that these patients actually didn’t fare so well.
#4. Lack of defined sample size or stopping rule
Most clinical trials have some pre-determined sample size. Interim analyses may be performed to evaluate for safety. If the study is stopped prior to reaching the goal size, this is frowned upon (as it may introduce an element of bias to the statistical analysis).
This study has no apparent power, or sample size, or anything. It’s undefined as to who decided to stop recruiting for the study or why. If data was being continuously analyzed, it’s possible that the study was terminated at a time when random trends in the data would favor Remdesivir.
#5. Lack of a primary endpoint
Most clinical trials also have a primary endpoint. We could debate whether over-reliance on the primary endpoint is problematic, but nonetheless – having pre-defined primary and secondary endpoints is important to prevent cherry-picking endpoints in a post-hoc fashion.
This study doesn’t seem to have any pre-defined endpoints at all. This allowed the authors to evaluate numerous endpoints and focus on whatever data they wanted to.
#6. Lack of information about the subjects
The baseline data about subjects is very scanty. For example, there is no information about disease biomarkers (e.g. D-dimer, LDH, C-reactive protein, ferritin). There are no markers of global physiologic severity (e.g. SOFA score or APACHE-II scores). Compared to most high-quality retrospective cohort series, this table is notable for its lack of information. In some cases (e.g. absolute lymphocyte count) – we know that this data was collected based on the study protocol – why was this basic data not presented? (My guess is that this data would have shown that the patients weren’t very sick at baseline).
#7. Inclusion of many patients who weren’t very ill
The initial inclusion criterion was anyone saturating 94% or less on room air. This allows the study to include some patients who were minimally ill. Nineteen patients who were included were not intubated, including 2 patients on room air and 10 patients on low-flow nasal cannula. These patients probably would have done well regardless of treatment. Some may not have even warranted hospital admission at all.
#8. Delayed administration of Remdesivir
Remdesivir was administered a median of 12 days after the onset of symptoms (interquartile range of 9-15). This delay may have been partly due to patient application for the drug and selection of patients.
Claiming benefit from a drug administered 12 days after disease onset is dubious for two reasons:
- The natural course of COVID is generally to deteriorate before day #12. So if you select for a group of patients who are still doing OK on day #12, the patients are likely to continue doing fine.
- By 12 days after symptom onset, viremia is generally beginning to decrease (due to immune activation). In fact, most of the clinical illness seen after 12 days may relate more to pathological hyperinflammation, not to direct viral damage. Thus, it doesn’t make sense that an antiviral drug administered this late in the disease course would have much clinical efficacy.
#9. Outcomes weren’t that awesome
Outcomes are shown above:
- Patients on room air or low-flow oxygen did fine (no surprise there).
- Among patients on non-invasive support, 5/7 did well but 1 died and 1 remained intubated at study end (which is about what might be expected for patients who survived off a ventilator for ~12 days).
- Among intubated patients, half did well, but the remainder either died or remained on ventilatory support.
These outcomes are pretty good (i.e., 50% extubation rate). Given the presence of considerable selection bias at every step along the way, they don’t represent anything to get too excited about.
#10. Possible signs of harm are obscured without a control group
23% of patients had serious adverse events (including multi-organ failure, septic shock, acute kidney injury, and hypotension)(table below). Are these true adverse events due to remdesivir? Or are these simply due to COVID infection? Without a control group, it’s anyone’s guess.
The authors concluded that “no new safety signals were detected during short-term remdesivir therapy.” I suppose that it technically correct. Based on the design of this trial, it’s impossible to say anything about the safety of remdesivir at all.
#11. Heavy involvement of pharma
Patient selection was performed by Gilead. Funding was provided by Gilead. Subsequently, the manuscript was largely written by professional writers employed by Gilead.
Over fifty doctors are listed as the co-authors. This author pool obviously has the talent and knowledge to draft an original manuscript. Why couldn’t they write the manuscript on their own, without professional help? Gilead probably didn’t want them to write an original and bias-free manuscript.
- This publication is grossly flawed, with many sources of bias. Overall, it reveals no meaningful information about Remdesivir.
- Like any new medication with unclear side-effects, Remdesivir should be subjected to RCTs.
- Gilead is a multi-billion-dollar pharmaceutical company, with ample resources. We are in the midst of a pandemic, with an abundance of COVID patients. There is no excuse for Gilead not to perform a well-powered, double-blind RCT. Hopefully such studies will be forthcoming shortly.
- Until an RCT is performed, further compassionate use of remdesivir probably isn't justified. This may simply represent a distraction when managing these extremely complex patients.
Image credit: Photo by Jossuha Théophile on Unsplash
- PulmCrit: How to quickly create a useful professional account in BlueSky - November 28, 2024
- PulmCrit Wee: Why MedTwitter should move to Bluesky - November 15, 2024
- PulmCrit Wee – A better classification of heart failure (HFxEF-RVxEF) - August 26, 2024
Very nice review. I wonder how NEJM accept this kind of manuscript
Overall, this is a reasonable analysis. Unfortunately, in his point #11 the author tips his own sort of bias. He should know that Gilead in fact is sponsoring a number of randomized controlled clinical trials, the results of which should be available within the next few months. Furthermore, the company itself has made no claims for this study and has specifically pointed to the need for the data from the other ongoing trials. In this time of crisis, it is not unreasonable that Gilead worked to provide the available information in a timely manner. It’s worth noting that they did… Read more »
Link to Science article: https://www.sciencemag.org/news/2020/01/fda-and-nih-let-clinical-trial-sponsors-keep-results-secret-and-break-law#
Ummm – I have no idea how this was accepted by NEJM [they have in the past rejected much higher quality work that I submitted to them!!!!] – Josh’s blog is pretty much what I would have written had I been asked to review this submission. I have no issues with commercial sponsorship of high quality research [openly declared] but this was NOT high quality research and adds nothing to guide decision clinical decision making. All I can conclude is that it provided ‘click bait’ to draw viewers to the journal (and subsequent citations to maintain their IF)
Me parece una descripción injusta, he seguido su trabajo sobre ventilación en pacientes covid y sus twitt son influyentes sobre las desiciones que se toman sobre pacientes críticos. Mencionan que aún faltan datos para concluir que el manejo ventilatorio del paciente crítico sea adecuado, pero lo publican y lo discuten sin darse cuenta que hoy los médicos leemos mucho en twitter. Me parece una contradicción criticar y decir que no ayud este estudio y a la vez publicar sus datos de investigación en twitter. Gran trabajo el que están realizando, pero considero que existe una contradicción de posturas frente a… Read more »
Fantastic review, Josh! Thanks for your diligent analysis helping add clarity to the mess of information out there.
Your reasoning is all correct. Prof Simon Maxwell, Professor of Clinical Pharmacology, University of Edinburgh, even calls NEJM’s study unethical. I tell you what is unethical: If everyday 10 000 people die from Covid-19, we have a drug that most probably helps, and we do not apply it in patients. Let us continue to do studies. But we have to use Remdesivir here and now.
Remdesivir isn’t a parachute. It failed in it’s designed role in Ebola and its only vulture behavior by Gilead the we are even having this conversation. There’s no a priori reason why Remdesivir should work. Theres the potential to do harm as well as provide a drug that doesn’t work that costs 5 grand an administration.
Remdesivir isn’t insulin in DKA where you can see the effects before your eyes. It’s more like niacin in hyperlipidemia for CAD prevention. It has a decent MOA but studies are needed to actually show benefit.
One can spin a set of data to one’s preference. This is like looking at glass half empty vs half full. The author was trying to poke holes into this set of circumstantial data, which is meaningless by itself. I agree the article in the NEJM isn’t telling much except the drug is “hopeful” by opinion. To me, this is a good enough of an observation. There’s nothing more to add.
While many articles such as this discuss the flaws in big pharma’s attempts at a treatment, a small biotech, CytoDyn, has an investigational drug, leronlimab, that is showing fantastic efficacy. This drug is now in FDA Phase 2 trails for mild to moderate infections and Phase 2b/3 for serious illness. It appears to be batting 1000 against the cytokine storm and is restoring patient’s immune response to fight the infection. Several patients have been discharged within only a few days of treatment.
To the person that thumbed down my comment, Please read this piece from Targeted Oncology.
https://www.targetedonc.com/news/updates-surrounding-leronlimab-use-in-patients-with-covid19-appear-promising
Pretty much agree with all points, although I don’t believe there is any hidden agenda behind the publication. It’s value is very limited, essentially anecdotal, similar to the original French study with hydroxychloroquin, pretty much no clinical relevance..
Can we just post the URL to this blogpost as a letter to the editor of NEJM?
Dear Josh,
Thank you for a thoughtful review. I also appreciate the comments and points of view below.
I would like to add a twelfth criticism for consideration: Why were only two or more patients required for an event to be considered adverse? So, if one patient developed anaphylaxis it would not be listed. Given the small study size, even one adverse event would be significant.
This is not even an article, its more of a “letter to the editor” . Absolutely shameful for NEJM. Just because we are desperate for news doesn’t mean we should settle for bottom of the barrel stuff
Excellent points as usual. The publication push and some of the “studies” ending up in high impact journals at present due to covid are unfortunately driving some peoples’ practice without actually reading the studies or critically thinking. At least if they aren’t going to read them and think critically here is a forum unpacking crucial weaknesses. Thanks!
Great job Josh. Greetings from many trainee in Anesthesia and Intensive care from Romania “Therapeutic remdesivir treatment initiated 12 hours post-inoculation reduced clinical signs, virus replication in the lungs, and decreased the presence and severity of lung lesions. A recent case series of 53 patients with severe COVID-19 pneumonia who received remdesivir under a compassionate-use protocol reported clinical improvement in 68% after a median follow-up of 18 days, with 13% mortality and a generally acceptable toxicity profile [62]. However, there was no comparison group of similar patients who received standard care at the participating institutions. Because RCTs for remdesivir have… Read more »
I’m not a doctor, but as an engineer I do understand statistics and data, and this all rings completely true. Particularly in light of today’s StatNews article which claims to publish “leaked” results from this study but this time out of U. Chicago. The way the article was written, combined with the results they suggest told me something wasn’t right. This insightful review of the NEJM paper confirms that belief. The suggestions and “anecdotes” in Stat are so completely out of line with even the data in NEJM (which as noted here, aside from suffering from bias, isn’t all that… Read more »
If anyone had any doubt that CNBC and Big Pharma are in bed together to manipulate the public this pretty much seals the deal. Look how Cytodyn is being treated relative to this “trial.” Another in the infinite number of reasons why so many people a) hate the healthcare system, b) don’t trust the media, and c) don’t trust Big Pharma.
No reason to publish such data They confuse the public No science at all Random administration of the drug
No criteria for response Arbitrary results What the Authors say for that?
During normal times, a study like this would have never seen the light of day. However, special times call for special standards! This is what I would call as “wartime science.” We sacrifice rigor to save lives. At least, that’s the hope. Having defended the current study (if you can call it a defense!), let me now be brutally honest. The title of the article is sleazy – including the word “compassionate” in it, so as to evoke compassion in us, the readers! This word also is used as an implicit justification for not including a control group – which… Read more »
I do have a bias toward HCQ+zinc+AZ for COVID-19. Thank you for the great article! I posted a summary and link in the forum. Amazing they are getting so much distance out of a drug that the NIH study characterized as resulting in no statistically significant reduction in mortality. But if you think th NEJM published “study” you cite was bad, check this ridiculous NEJM article by 24 “professional” authors that were allegedly trying HCQ as prophylaxis – but administered HCQ for only 5 consecutive days! (rather than once a week for 2 weeks as label suggests, or for COVID… Read more »
Today I was searching for a Health Blog which gives me detailed information about <a href=”https://www.nash24x7.com/whats-non-alcoholic-fatty-liver-diseases/”>Nonalcoholic Fatty Liver Disease</a> and <a href=”https://www.nash24x7.com/fat-in-liver/”>Fat in liver</a> and finally I find this awesome content with helping material. So I also recommend you to visit <a href=”https://www.nash24x7.com/”>Nash24x7</a> for reading More.
Quality content is the crucial to attract the viewers
to go to see the site, that’s what this website is providing. Nafld