Sunday, April 6, 2014

Underperforming the Market: Why Researchers are Worse than Professional Stock Pickers and A Way Out

I was reading in the NYT yesterday a story about Warren Buffet and how the Oracle of Omaha has trailed the S&P 500 for four of the last five years.  It was based on an analysis done by a statistician who runs a blog called Statistical Ideas, which has a post on p-values that links to this Nature article a couple of months back that describes how we can be misled by P-values.  And all of this got me thinking.

We have a dual problem in medical research:  a.)  of conceiving alternative hypotheses which cannot be confirmed in large trials free of bias;  and b.) not being able to replicate the findings of positive trials.  What are the reasons for this?

Tuesday, April 1, 2014

Absolute Confusion: How Researchers Mislead the Public with Relative Risk

This article in Sunday's New York Times about gauging the risk of autism highlights an important confusion in the appraisal of evidence from clinical trials and epidemiological studies that appears to be shared by laypersons, researchers, and practitioners alike:  we focus on relative risks when we should be concerned with absolute risks.

The rational decision maker, when evaluating a risk or a benefit, is concerned with the absolute magnitude of that risk or benefit.  A proportional change from an arbitrary baseline (a relative risk) is irrelevant.  Here's an example that should bring this into keen focus.

If you are shopping and you find a 50% off sale, that's a great sale.  Unless you're shopping for socks.  At $0.99 a pair, you save $0.50 with that massive discount.  Alternatively, if you come across a 3% sale, but it's at the Audi dealership, that paltry discount can save you $900 on a $30,000 Audi A4.   Which discount should you spend the day pursuing?  The discount rate mathematically obscures the value of the savings.  If we framed the problem in terms of absolute savings, we would be better consumers.  But retailers know that saying "50% OFF!" attracts more attention than "$0.50 OFF!" in the sock department.  Likewise, car salesmen know that writing "$1000 BELOW INVOICE!" on the windshield looks a lot more attractive than "3% BELOW INVOICE!"

Sunday, March 23, 2014

Lost Without a MAP: Blood Pressure Targets in Septic Shock

Another of the critical care articles published early online at last week was this trial of High versus Low Blood-Pressure Target in Patients with Septic Shock.  In this multicenter, open-label trial, the authors enrolled 776 patients in France and randomized them to a target MAP (mean arterial pressure) of 65-70 mm Hg (low target) versus 80-85 (high target).  The hypothesis is that a higher pressure, achieved through vasopressor administration, will improve 28-day mortality.  If you don't already know the result, guess if the data from this trial support or confirm the hypothesis (the trial had 80% power to show a 10% absolute reduction in mortality).

Thursday, March 20, 2014

Sepsis Bungles: The Lessons of Early Goal Directed Therapy

On March 18th, the NEJM published early online three original trials of therapies for the critically ill that will serve as fodder for several posts.  Here, I focus on the ProCESS trial of protocol guided therapy for early septic shock.  This trial is in essence a multicenter version of the landmark 2001 trial of Early Goal Directed Therapy (EGDT) for severe sepsis by Rivers et al.  That trial showed a stunning 16% absolute reduction in mortality in sepsis attributed to the use of a protocol based on physiological goals for hemodynamic management.  That absolute reduction in mortality is perhaps the largest for any therapy in critical care medicine.  If such a reduction were confirmed, it would make EGDT the single most important therapy in the field.  If such reduction cannot be confirmed, there are several reasons why the Rivers results may have been misleading:

There were other concerns about the Rivers study and how it was later incorporated into practice, but I won't belabor them here.  The ProCESS trial randomized about 1350 patients among three groups, one simulating the original Rivers protocol, one to a modified Rivers protocol, and one representing "standard care" that is, care directed by the treating physician without a protocol.  The study had 80% power to demonstrate a mortality reduction of 6-7%.  Before you read further, please wager, will the trial show any statistically significant differences in outcome that favor EGDT or protocolized care?

Friday, February 28, 2014

Overediagnosis and Mitigated Overdiagnosis: Ongoing problems with Breast and Lung Cancer Screening

I got to thinking about cancer screening (again) in the last week after reading in BMJ about the Canadian National Breast Screening Study (CNBSS).  That article piqued my interest because I immediately recalled the brouhaha that ensued after the U.S. Preventative Services Task Force (USPSTF) recommended that women not get mammograms until  age 50 rather than age 40.  That uproar was similar to the outcry by urologists when the USPSTF recommended against screening for prostate cancer with PSA testing.  Meanwhile, changes in the cholesterol guidelines have incited intellectual swashbuckling among experts in that field.  Without getting into the details, observers of these events might generate the following hypotheses:
  1. People are comfortable with the status quo and uncomfortable with change
  2. People get emotionally connected to good causes and this makes the truth blurry, or invisible.  After you've participated in the Race for the Cure, it's hard to swallow the possibility that the linchpin of the Race might not be as useful as we thought; and is no longer recommended for a whole swath of women. 
  3. People are terrified of cancer
  4. Screening costs money.  Somebody pockets that money.  Urologists and radiologists and gastroenterologists LOVE screening programs.  So do Porche dealers.

Monday, February 10, 2014

Brief Updates on Hypothermia, Hyperglycemia, Cholesterol, Blood Pressure Lowering in Stroke and Testosterone

I've read a lot of interesting articles recently, but none that are sufficient fodder for a dedicated post.  So here I will update some themes from previous blog posts with recent articles from NEJM and JAMA that relate to them.

Prehospital Induction of Hypothermia After Cardiac Arrest
In this article in the January 1st issue of JAMA, investigators from King County Washington report the results of a trial which tested the hypothesis that earlier (prehospital) induction of hypothermia, by infusing cold saline, would augment the assumed benefit of hypothermia that is usually initiated in the hospital for patients with ventricular fibrillation.  Please guess what was the effect of this intervention on survival to hospital discharge and neurological outcomes.

You were right.  There was not even a signal, not a trend towards benefit, even though body temperature was lower by 1 degree Celcius and time to target hypothermia temperature in the hospital was one hour shorter.  However, the intervention group experienced re-arrest in the field significantly more often than the control group and had more pulmonary edema and diuretic use.  Readers interested in exploring this topic further are referred to this post on Homeopathic Hypothermia.

Hyperglycemic Control in Pediatric Intensive Care
In this article in the January 9th issue of NEJM, we are visited yet again by the zombie topic that refuses to die.  We keep looking for subgroups or populations that will benefit, and if we find one that appears to, it will be a Type I error, thinks the blogger with Bayesian inclinations.  In this trial, 1369 pediatric patients at 13 centers in England were randomized to tight versus conventional glycemic control.  Consistent with other trials in other populations, there was no benefit in the primary outcome, but tightly "controlled" children had much more and severe hypoglycemia.  The "cost effectiveness" analysis they report is irrelevant.  You can't have "cost effectiveness" of an ineffective therapy.  My, my, how we continue to grope.

Wednesday, January 29, 2014

Does Investigating Delirium Make You Delirious? A Sober Look at Sedation and Analgesia in the ICU

Michael's Milk
I rarely use the Medical Evidence Blog to discuss review articles, but today's NEJM has one that I can't pass up about Sedation and Delirium in the Intensive Care Unit.  It is my opinion that we have gotten carried away by the torrent of articles, many in prominent journals, about delirium in the ICU and that while this is an important topic for research, it is extremely premature to try to translate the findings into practice, and moreover, that the approach to sedation suggested by the article is lacking in common sense.

As chronicled in the accompanying perspective article by D.S. Jones, delirium has been around as long as ICUs have, and the longer you're there, the more likely you will become delirious.  It's an exposure thing.  Thus, until somebody reports the results of a trial of delirium treatment or prevention that has important and undeniable effects on clinically relevant outcomes, I will continue to approach delirium as I always have - by going to great lengths to get patients out of bed, off the vent, and out of the ICU as fast as I possibly can - because these things benefit all patients regardless of whether they have an impact on delirium.

Thursday, January 23, 2014

White Noise: Trials of Pharmaceuticals for Alzheimer's Disease

"But we are not here concerned with hopes or fears, only with the truth as far as our reason allows us to discover it." - Charles Darwin

In yesterday's NEJM, the results of two trials of antiamyloid monoclonal antibodies (sonalezumab and bapeneuzumab)  for Alzheimer's Disease (AD) are published.  I became interested in the evidence for AD treatments after the recent trial of Vitamin E and Mematine for AD (the TEAM-AD VA Cooperative Trial) was published in JAMA earlier this month.  Regular readers know that I think that the prior probability that vitamins, minerals, and antioxidants are beneficial for any disease outside of deficiency states is very low.  The vitamin E trial was the impetus for some background investigation which I will summarize below.

Friday, December 27, 2013

Billions and Billions of People on Statins? Damn the Torpedos and Full Speed Ahead

Absolutely Relative
Risk is in the Mind of the Taker
Among the many editorials providing background and backlash about the new cholesterol guidelines is this one:  More Than a Billion People Taking Statins? by John Ioannidis, which echoes the worries of others that the result of the guidelines (which changed the 10-year risk threshold for treatment from 10% to 7.5%) may be that many more people (billions and billions?) will be prescribed statins.  But the title is a curious one - if statins are beneficial, should we lament their widespread prescription and adoption or is it just unfortunate that heart disease is so prevalent? Whose side are we on, the cure or the disease?

Are the premises of the guidelines flawed leading to flawed extrapolations, or are the premises correct and we just don't like the implications?  Let's look at the premises - because if they're flawed, we may find that other premises we have accepted are flawed.

Wednesday, November 20, 2013

Chill Out: Homeopathic Hypothermia after Cardiac Arrest

In the Feb 21, 2002 NEJM, two trials of what came to be known as therapeutic hypothermia (or HACA - Hypothermia after Cardiac Arrest) were simultaneously published:  one by the HACA study group and another by Bernard et al.  During the past decade, I can think of only one other therapy which has caused such a paradigm shift in care in the ICU:  Intensive Insulin Therapy (ill-fated as it were).  Indeed, even though the 2002 studies specifically limited enrollment to out of hospital (OOH) cardiac arrest with either Ventricular Tachycardia (VT) or Ventricular Fibrillation (VF), the indications have been expanded at many institutions to include all patients with coma after cardiac arrest regardless of location or rhythm (or any other original exclusion criterion), so great has been the enthusiasm for this therapy, and so zealous its proponents.

Readers of this blog may know that I harbor measured skepticism for HACA even though I recognize that it may be beneficial.  From a pragmatic perspective, it makes sense to use it, since the outcome of hypoxic-ischemic encephalopathy (HIE) and ABI (Anoxic Brain Injury) is so dismal.  But what did the original two studies actually show?
  • The HACA group multicenter trial randomized 273 patients to hypothermia versus control and found that the hypothermia group had higher rates of "favorable neurological outcome" (a cerebral performance category of 1 or 2 - the primary endpoint) with RR of 1.40 and 95% CI 1.08-1.81; moreover, mortality was lower in the hypothermia group, with RR 0.74 and 95% CI 0.58-0.95
  • The Bernard et al study randomized 77 patients to hypothermia versus control and found that survival (the primary outcome) was 49% and 26% in the hypothermia and control groups, respectively, with P=0.046

Monday, November 18, 2013

Dead in the Water: Colloids versus Crystalloids for Fluid Resuscitation in the ICU

It is a valid question:  at what point has a concept been tested ad infinitum such that further testing is not worthwhile?  There are at least three reasons why additional study of a concept may not be justified:

  1. Because the prior probability of success is so low (based on extant trials) that a subsequent trial is unlikely to influence the posterior probability that any success represents the truth.  (This is a Bayesian or meta-analytic worldview.)
  2. Because the low probability of success does not justify the expense of additional trials
  3. Because the low probability of success violates bioethical precepts mandating that trials must have added value for patients and society
And so we have, in the November 6th edition of JAMA, the CRISTAL trial of colloids versus crystalloids for resuscitation in the ICU.  As is customary, I will leave it to interested readers to peruse the manuscript for details.  My task here is to provide some background and nuance.

Saturday, November 16, 2013

The Cardiologist Giveth, then the Cardiologist Taketh Away: Revision of the Cholesterol Guidelines

There has been quite a stir this week with the publication of the newest revision of the ACC/AHA guidelines for the treatment of cholesterol.  The New York Times is awash with articles summarizing or opining on the changes and many of the authors are perspicacious observers:
As the old Spanish proverb states, "rio revuelto, ganancia de pescadores" - when the river is stirred up, the fishermen benefit.  I will admit that I'm gloating a bit since I consider the new guidelines to be a tacit affirmative nod to several posts on the topic of the cholesterol hypothesis (CH).  (More posts here and here and here, among several others - search for "cholesterol" or "causal pathways" on the Medical Evidence Blog search bar.)

Sunday, November 3, 2013

The Intensivist Giveth Then the Intensivist Taketh Away: Esmolol in Septic Patients Receiving High Dose Norepinephrine

Two studies in the October 23/30 issue of JAMA serve as fodder for reflection on the history and direction of critical care research and the hypotheses that drive it.   Morelli et all report the results of a study of Esmolol in septic shock.  To quickly summarize, this was a single center dose ranging study the primary aim of which was to determine if esmolol could be titrated to a heart rate goal (primary outcome), presumably with the later goal of performing a phase 3 clinical trial to see if esmolol, titrated in such a fashion, could favorably influence clinical outcomes of interest.  154 patients with septic shock on high dose norepinephrine with a heart rate greater than 95 were enrolled, and heart rate was indeed lower in the esmolol group (P less than 0.001).  Perhaps surprisingly, hemodynamic parameters, lactate clearance, and pressor and fluid requirements were (statistically significantly) improved in the esmolol group.  Most surprising (and probably the reason why we find this published in JAMA rather than Critical Care Medicine - consider that outlier results such as this may get disproportionate attention), mortality in the esmolol group was 50% compared to 80% in the control group (P less than 0.001).  The usual caveats apply here:  a small study, a single center, lack of blinding.  And regular readers will guess that I won't swallow the mortality difference.  I'm a Bayesian (click here for a nice easy-to-use Bayesian calcluator), there's no biological precedent for such a finding and it's too big a bite for me to swallow. So I will go on the record here as stating that I'm betting against similar results in a larger trial.

I'm more interested in how we formulate the hypothesis that esmolol will provide benefit in septic shock.  I was a second year medical student in 1995 when Gattinoni et al published the results of a trial of "goal-oriented hemodynamic therapy" in critically ill patients in the NEJM.  I realize that critical care research as we now recognize it was in its adolescence then, as a quick look at the methods section of that article demonstrates.  I also recognize that they enrolled a heterogenous patient population.  But it is worth reviewing the wording of the introduction to their article:

Recently, increasing attention has been directed to the hemodynamic treatment of critically ill patients, because it has been observed in several studies that patients who survived had values for the cardiac index and oxygen delivery that were higher than those of patients who died and, more important, higher than standard physiologic values.1-3 Cardiac-index values greater than 4.5 liters per minute per square meter of body-surface area and oxygen-delivery values greater than 650 ml per minute per square meter — derived empirically on the basis of the median values for patients who previously survived critical surgical illness — are commonly referred to as supranormal hemodynamic values.4

Saturday, October 12, 2013

Goldilocks Meets Walter White in the ICU: Finding the Temperature (for Sepsis and Meningitis) that's Just Right

In the Point/Counterpoint  section of the October issue of Chest, two pairs of authors spar over whether fever should be controlled in sepsis by either pharmacological or external means.  Readers of this blog may recall this post wherein I critically appraised the Schortgen article on external cooling in septic shock that was in AJRCCM last year.  Apparently that article made a more favorable impression on some practitioners than it did on me, as the proponents of cooling in the Chest piece hang their hats on this article (and their ability to apply physiological principles to medical therapeutics).  (My gripes with the Schortgen study were many, including a primary endpoint that was of little value, cherrypicking the timing of the secondary mortality endpoint, and the lack of any biological precedent for manipulation of body temperature improving mortality in any disease.)

Reading the Point and Counterpoint piece (in addition to an online first article in JAMA describing a trial of induced hypothermia in severe bacterial meningitis - more on that later) allowed me to synthesize some ideas about the epistemology (and psychology) of medical evidence and its evaluation that I have been tossing about in my head for a while.  Both the proponent pair and the opponent pair of authors give some background physiological reasoning as to why fever may be, by turns, beneficial and detrimental in sepsis.  The difference, and I think this is typical, is that the proponents of fever reduction:  a.) seem much more smitten by their presumed understanding of the underlying physiology of sepsis and the febrile response; b.) focus more on minutiae of that physiology; c.) fail to temper their faith in application of physiological principles with the empirical data; and d.) grope for subtle signals in the empirical data that appear to rescue the sinking hypothesis.

Friday, August 2, 2013

Sause for the Goose, Sauce for the Gander: Low Tidal Volume Ventilation in the Operating Theatre

PIBW is based on height, not weight.
Following my usual procedure, I read the title and abstract of the methods of this article on Intraoperative Low Tidal Volume Ventilation in this week's NEJM, and I made a wager with myself on what the outcome would be.  Because there are both biological plausibility and biological precedent for low tidal volume, and because it is one of the few interventions in critical care in which I have supreme confidence (yes, you can conclude that I'm biased), my prior probability for this intervention is high and I wagered that the study would be positive.  If you have not already done so, read the methods in the abstract and make your own wager before you read on.

This trial is solid but not bombproof.  Outcomes assessors were blinded and so were post-operative care providers, but anesthesiologists administering tidal volumes were not.  Outcomes themselves, while mostly based on consensus definitions (sometimes a consensus of collective ignorance), are susceptible to ascertainment and misclassification biases.  The outcome was a composite, something that I like, as will be elaborated in a now published letter in AJRCCM.  A composite outcome allows an additive effect between component outcomes and effectively increases study power.  This is essential in a study such as this, where only 400 patients were enrolled and the study had "only" 80% power to detect a reduction in the primary outcome from 20% to 10%.  As we have shown, detecting a difference of this magnitude in mortality is a difficult task indeed, and most critical care studies seeking such a difference are effectively underpowered.  How many effective (in some aspect other than mortality) therapies have been dismissed because of this systemic underpowering in critical care research is anybody's guess.

Thursday, June 20, 2013

More is Not Less, It Just Costs More: Early Tracheostomy, Early Parenteral Nutrition, and Rapid Blood Pressure Lowering in ICH

The past 2 weeks have provided me with some interesting reading of new data that deserve to be integrated with several other studies and themes discussed in this blog.  The three trials below share the goal of intervening early and aggressively so I thought it may be interesting to briefly consider them together.

Firstly, Young et al (May 22/29, 2013 issue of JAMA) report the results of the TracMan multicenter trial of early tracheostomy in ICUs in the UK.  These data seal the deal on an already evolving shift in my views on early tracheostomy that were based on anecdotal experience and earlier data from Rumbak and Terragni.  Briefly, the authors enrolled 899 patients expected to receive at least 7 more days of mechanical ventilation (that prediction was no more reliable in the current trial than it had been in previous trials) and randomized them to receive a trach on day 4 (early) versus on day 10 (late).    The early patients did end up receiving less sedatives and  had a trend toward shorter duration of respiratory support.  But their KM curves are basically superimposable and the mortality rates virtually identical at 30 days.  These data, combined with other available studies, leave little room for subjective interpretation.  Early tracheostomy, it is very likely, does not favorably affect outcomes enough to justify its costs and risks.

Friday, May 31, 2013

Over Easy? Trials of Prone Positioning in ARDS

Published May 20 in the  NEJM to coincide with the ATS meeting is the (latest) Guerin et al study of Prone Positioning in ARDS.  The editorialist was impressed.  He thinks that we should start proning patients similar to those in the study.  Indeed, the study results are impressive:  a 16.8% absolute reduction in mortality between the study groups with a corresponding P-value of less than 0.001.  But before we switch our tastes from sunny side up to over easy (or in some cases, over hard - referred to as the "turn of death" in ICU vernacular) we should consider some general principles as well as about a decade of other studies of prone positioning in ARDS.

First, a general principle:  regression to the mean.  Few, if any, therapies in critical care (or in medicine in general) confer a mortality benefit this large.  I refer the reader (again) to our study of delta inflation which tabulated over 30 critical care trials in the top 5 medical journals over 10 years and showed that few critical care trials show mortality deltas (absolute mortality differences) greater than 10%.   Almost all those that do are later refuted.  Indeed it was our conclusion that searching for deltas greater than or equal to 10% is akin to a fool's errand, so unlikely is the probability of finding such a difference.  Jimmy T. Sylvester, my attending at JHH in late 2001 had already recognized this.  When the now infamous sentinel trail of intensive insulin therapy (IIT) was published, we discussed it at our ICU pre-rounds lecture and he said something like "Either these data are faked, or this is revolutionary."  We now know that there was no revolution (although many ICUs continue to practice as if there had been one).  He could have just as easily said that this is an anomaly that will regress to the mean, that there is inherent bias in this study, or that "trials stopped early for benefit...."

Monday, May 20, 2013

It All Hinges on the Premises: Prophylactic Platelet Transfusion in Hematologic Malignancy

A quick update before I proceed with the current post:  The Institute of Medicine has met and they agree with me that sodium restriction is for the birds.  (Click here for a New York Times summary article.)  In other news, the oh-so-natural Omega-3 fatty acid panacea did not improve cardiovascular outcomes as reported in the NEJM on May 9th, 2013.

An article by the TOPPS investigators in the May 9th NEJM is very useful to remind us not to believe everything we read, to always check our premises, and that some data are so dependent on the perspective from which they're interpreted or the method or stipulations of analysis that they can be used to support just about any viewpoint.

The authors sought to determine if a strategy of withholding prophylactic platelet transfusions for platelet counts below 10,000 in patients with hematologic malignancy was non-inferior to giving prophylactic platelet transfusions.  I like this idea, because I like "less is more" and I think the body is basically antifragile.  But non-inferior how?  And what do we mean by non-inferior in this trial?

Saturday, April 27, 2013

Tell Them to Go Pound Salt: Ideology and the Campaign to Legislate Dietary Sodium Intake

In the March 28th, 2013 issue of the NEJM, a review of sorts entitled "Salt in Health and Disease - A Delicate Balance" by Kotchen et al can be found.  My interest in this topic stems from my interest in the question of association versus causation, my personal predilection for salt, my observation that I lose a good deal of sodium in outdoor activities in the American Southwest, and my concern for bias in the generation of and especially the implementation of evidence in medicine as public policy.

This is an important topic, especially because sweeping policy changes regarding the sodium content of food are proposed, but it is a nettlesome topic to study, rife with hobgoblins.  First we need a well-defined research question:  does reduction in dietary sodium intake:  a.) reduce blood pressure in hypertensive people?  in all people?  b.) does this reduction in hypertension lead to improved outcomes (hypertension is in some ways a surrogate marker)?  In a utopian world, we would randomize thousands of participants to diets low in sodium and "normal" in sodium, we would measure sodium intake carefully, and we would follow the participants for changes in blood pressure and clinical outcomes for a protracted period.  But alas, this has not been done, and it will not likely be done because of cost and logistics, among other obstacles (including ideology).

Friday, April 19, 2013

David versus Goliath on the Battlefield of Non-inferiority: Strangeness is in the Eye of the Beholder

In this week's JAMA is my letter to the editor about the CONSORT statement revision for the reporting of non-inferiority trials, and the authors' responses.  I'll leave it to interested readers to view for themselves the revised CONSORT statement, and the letter and response.

In sum, my main argument is that Figure 1 in the article is asymmetric, such that inferiority is stochastically less likely than superiority and an advantage is therefore conferred to the "new" [preferred; proprietary; profitable; promulgated] treatment in a non-inferiority trial.  Thus the standards for interpretation of non-inferiority trials are inherently biased.  There is no way around this, save for revising the standards.

The authors of CONSORT say that my proposed solution is "strange" because it would require revision of the standards of interpretation for superiority trials as well.  For me it is "strange" that we would endorse asymmetric and biased standards of interpretation in any trial.  The compromised solution, as I suggested in my letter, is that we force different standards for superiority only in the context of a non-inferiority trial.  Thus, superiority trial interpretation standards remain untouched.  It is only if you start with a non-inferiority trial that you have a higher hurdle to claiming superiority that is contingent on evidence of non-inferiority in the trial that you designed.  This would disincentivise the conduct of non-inferiority trials for a treatment that you hope/think/want to be superior.  In the current interpretation scheme, it's a no-brainer - conduct a non-inferiority trial and pass the low hurdle for non-inferiority, and then if you happen to be superior too, BONUS!

In my proposed scheme, there is no bonus superiority that comes with a lower hurdle than inferiority.  As I said in the last sentence, "investigators seeking to demonstrate superiority should design a superiority trial."  Then, there is no minimal clinically important difference (MCID) hurdle that must be cleared, and a statistical difference favoring new therapy by any margin lets you declare superiority.  But if you fail to clear that low(er) hurdle, you can't go back and declare non-inferiority.  

Which leads me to something that the word limit of the letter did not allow me to express:  we don't let unsuccessful superiority trials test for non-inferiority contingently, so why do we let successful non-inferiority trials test for superiority contingently?

Symmetry is beautiful;  Strangeness is in the eye of the beholder.

(See also:  Dabigatran and Gefitinib especially the figures, analogs of Figure 1 of Piaggio et al, on this blog.)

Wednesday, April 17, 2013

Out to Lunch: Nutrition and Supplementation in Critical Illness

A study in week's issue of the NEJM (Heyland et al, Glutamine in Critical Illness, April 18th, 2013) left me titillated in consideration of how new evidence demonstrates underlying misconceptions, shortcomings, and biases in our understanding of, and general approach to, disease and its pathophysiology.  Before you read on, try to predict:  Will supplemental glutamine and anti-oxidants influence the course of critical illness?

The Canadian Critical Care Trials group has continued the effort to determine the causal role of macro- and micronutrients and their deficiency and supplementation in critical (and other) illness.  The results are discouraging (glutamine and anti-oxidants don't work), but only if we consider RCTs to be a tool for the assessment of the therapeutic value of putative molecules and their manipulation in disease states.  RCTs are such a tool, but only if we happen to be fortunate enough to be pursuing a causal pathway.  In the absence of this good fortune, RCTs remain valuable but only to help us understand that the associations we have labored to delineate are not causal associations, and that we should direct our focus to other, potentially more fruitful, investigations.  As I articulated in the last post, this dual role of RCTs represents a paradox which can be the source of great cognitive dissonance (and misunderstanding).  The (properly conducted and adequately powered) RCT is a method for determining if observational associations are causal associations, but the promise of confirming causal associations in an RCT by manipulating dependent variables with a potential therapeutic agents carries with it the possibility of proving the efficacy of a disease treatment.   During this protracted scientific process, there is a tendency to get carried away, such that our hypothesis mutates into a premise that we are studying a causal factor and the RCT is the last hurdle to confirming that we have advanced not only the science of causation, but also clinical therapeutics.  Alas, the historical record shows that we are far better at advancing our understanding (if we are willing to accept the results for what they are) than we are at finding new treatments for disease, because most of the associations we are investigating turn out not to be causal.

Sunday, March 24, 2013

Why Most Clinical Trials Fail: The Case of Eritoran and Immunomodulatory Therapies for Sepsis

The experimenter's view of the trees.
The ACCESS trial of eritoran in the March 20, 2013 issue of JAMA can serve as a springboard to consider why every biological and immunomodulatory therapy for sepsis has failed during the last 30 years.  Why, in spite of extensive efforts spanning several decades have we failed to find a therapy that favorably influences the course of sepsis?  More generally, why do most clinical trials, when free from bias, fail to show benefit of the therapies tested?

For a therapeutic agent to improve outcomes in a given disease, say sepsis, a fundamental and paramount precondition must be met:  the agent/therapy must interfere with part of the causal pathway to the outcome of interest.  Even if this precondition is met, the agent may not influence the outcome favorably for several reasons:
  • Causal pathway redundancy:  redundancy in causal pathways may mitigate the agent's effects on the downstream outcome of interest - blocking one intermediary fails because another pathway remains active
  • Causal factor redundancy:  the factor affected by the agent has both beneficial and untoward effects in different causal pathways - that is, the agent's toxic effects may outweigh/counteract its beneficial ones through different pathways
  • Time dependency of the causal pathway:  the agent interferes with a factor in the causal pathway that is time dependent and thus the timing of administration is crucial for expression of the agent's effects
  • Multiplicity of agent effects:  the agent has multiple effects on multiple pathways - e.g., HMG-CoA reductase inhibitors both lower LDL cholesterol and have anti-inflammatory effects.  In this case, the agent may influence the outcome favorably, but it's a trick of nature - it's doing so via a different mechanism than the one you think it is.

Tuesday, March 12, 2013

Falling to Pieces: Hemolysis of the Hemoglobin Hypothesis

A paramount goal of this blog is to understand the evidence as it applies to the epistemology of medical knowledge, hypothesis testing, and overarching themes in the so-called evidence based medicine movement.  Swedberg et al report the results of a large[Amgen funded] randomized controlled trial of darbepoetin [to normalize hemoglobin values] in congestive heart failure (published online ahead of print this weekend) which affords us the opportunity to explore these themes afresh in the context of new and prior data.

The normalization heuristic, simply restated, is the tendency for all healthcare providers including nurses, respiratory therapists, nutritionists, physicians, and pharmacists among others, to believe intuitively or explicitly that values and variables that can be measured should be normalized if interventions to this avail are at their disposal.  As an extension, modifiable variables should be measured so that they can be normalized.  This general heuristic is deeply flawed, and indeed practically useless as a guide for clinical care.

Sunday, March 3, 2013

HFOV Fails as a Routine Therapy for moderate-to-severe ARDS. Musings on the Use and Study of “Rescue Therapies”.

Ferguson et al report the results of the OSCILLATE randomized controlled trial of HFOV for moderate to severe ARDS in this week’s NEJM.  (A similar RCT of HFOV, the OSCAR trial, is reported in the same issue but I limit most of my commentary to OSCILLATE because I think it’s a better and more interesting trial and more data are presented in its report.)  A major question is answered by this trial, but an important question remains open:  is HFOV an acceptable and rational option as “rescue therapy” in certain patients with “refractory” ARDS?  I remain undecided about this question, and its implications are the subject of this post.

Before I segue to the issue of the study and efficacy of rescue therapies, let’s consider some nuances of this trial:

·         Patients in both groups received high doses of sedatives (average midazolam dose for the first week: 8.3 mg/hour in the HFOV group versus 5.9 mg/hour in the control group – a 41% increase in HFOV).  Was this “too much” sedation?  What if propofol had been used instead?

·         Patients in the HFOV group received significantly more paralytics.  If you believe the Papazian data (I don’t) paralytics should confer a mortality benefit in early ARDS and this should contribute to LOWER mortality in the HFOV group.  What if paralytics had been used less frequently?

·         Does HFOV confer a nocebo effect by virtue of its “unnatural” pattern of ventilation, its “requirement” for more sedation and paralysis, or the noise associated with its provision, or its influence on the perceptions of caregivers and patient’s families (recognizing that deaths after withdrawal of life support were similar in HFOV versus conventional ventilation (55 versus 49%, P=0.12)?

·         The respiratory frequency in the HFOV group (5.5 Hz) was at the low end of the usual range (3-15 Hz).  If a higher frequency (and a lower tidal volume) had been delivered, would the result have changed?  (Probably not.)

·         What about the high plateau pressure in the control group (32 cm H2O) despite the low tidal volume of 6.1 ml/kg PBW?  Why was not tidal volume reduced such that plateau pressure was lower than the commonly recommended target of 30 cm H2O?  Did this make a difference?  (Probably not.)

·         Why was mortality higher in the minority (12%) of control patients who were changed to HFOV (71% mortality)?  Is this related to confounding by indication or reflective of the general harmful effects of HFOV?

·         Why was there a difference between the OSCILLATE study and the OSCAR study, reported in the same issue, in terms of mortality?  Because OSCILLATE patients were sicker?  Because OSCAR control patients received higher tidal volumes, thereby curtailing the advantage of conventional ventilation?  I find this last explanation somewhat compelling.

Monday, January 28, 2013

Coffee Drinking, Mortality, and Prespecified Falsification Endpoints

A few months back, the NEJM published this letter in response to an article by Freedman et al in the May 17, 2012 NEJM reporting an association between coffee drinking and reduced mortality found in a large observational dataset.  In a nutshell, the letter said that there was no biological plausibility for mortality reductions resulting from coffee drinking so the results were probably due to residual confounding, and that reductions in mortality in almost all categories (see Figure 1 of the index article) including accidents and injuries made the results dubious at best.  The positive result in the accidents and injuries category was in essence a failed negative control in the observational study.

Last week in the January 16th issue of JAMA Prasad and Jena operationally formalized this idea of negative controls for observational studies, especially in light of Ioannidis' call for a registry of observational studies.  They recommend that investigators mining databases establish a priori hypotheses that ought to turn out negative because they are biologically implausible.  These hypotheses can therefore serve as negative controls for the observational associations of interest, the ones that the authors want to be positive.  In essence, they recommend that the approach to observational data become more scientific.  At the most rudimentary end of the dataset analysis spectrum, investigators just mine the data to see what interesting associations they can find.  In the middle of the spectrum, investigators have a specific question that they wish to answer (usually in the affirmative), and they leverage a database to try to answer that question.  Prasad and Jena are suggesting going a step further towards the ideal end of the spectrum:  to specify both positive and negative associations that should be expected in a more holistic assessment of the ability of the dataset to answer the question of interest.  (If an investigator were looking to rule out an association rather than to find one, s/he could use a positive control rather than a negative one [a falsification end point] to establish the database's ability to confirm expected differences.)

I think that they are correct in noting that the burgeoning availability of large databases (of almost anything) and the ease with which they can be analyzed poses some problems for interpretation of results.  Registering observational studies and assigning prespecified falsification end points should go a long way towards reducing incorrect causal inferences and false associations.

I wish I had thought of that.

Added 3/3/2013 - I just realized that another recent study of dubious veracity had some inadvertent unspecified falsification endpoints, which nonetheless cast doubt on the results.  I blogged about it here:  Multivitamins caused epistaxis and reduced hematuria in male physicians.

Sunday, January 27, 2013

Therapeutic Agnosticism: Stochastic Dominance of the Null Hypothesis

Here are some more thoughts on the epistemology of medical science and practice that were stimulated by reading three articles this week relating to monitoring interventions:  monitoring respiratory muscle function in the ICU (AJRCCM, January 1, 2013); monitoring intracranial pressure in traumatic brain injury (NEJM, December 27, 2013); and monitoring of gastric residual volume in the ICU (JAMA, January 16th, 2013).

In my last post about transfusion thresholds, I mused that overconfidence in their understanding of complex pathophysiological phenomena (did I say arrogance?) leads investigators and practitioners to overestimate their ability to discern the value and efficacy of a therapy in medicine.  Take, for instance, the vascular biologist studying pulmonary hypertension who, rounding in the ICU, elects to give sildenafil to a patient with acute right heart failure, and who proffers a plethora of complex physiological explanations for this selection.  Is there really any way for anyone to know the effects of sildenafil in this scenario?

Monday, January 14, 2013

Death by 1000 Needlesticks: The Nocebo effects of Hospitalization

I couldn't decide if this belonged on Status Iatrogenicus or the Medical Evidence Blog.  Since it has relevance to both, I'll post a link here:

Hemoglobin In Limbo: How Low Can [should] It Go?

In this post about transfusion thresholds in elderly patients undergoing surgery for hip fracture, I indulged in a rant about the irresistible but dodgy lure of transfusing hospitalized patients with anemia (which I attributed to the normalization heuristic) and the wastefullness and potential harms it entails.  But I also hedged my bets, stating that I could get by with transfusing only one unit of blood a month in non-acutely bleeding patients, while noting in a comment that a Cochrane review of this population was equivocal and the authors suggested an RCT of transfusion in acute upper gastrointestinal hemorrhage.  Little did I know at the time that just such a trial was nearing completion, and that 12 units of PRBCs could probably get me by for a year in just about all the patients I see.

In this article by Villanueva in the January 3, 2013 issue of the NEJM, Spanish investigators report the results of a trial of transfusion thresholds in patients with acute upper gastrointestinal hemorrhage.  After receiving one unit of PRBCs for initial stabalization, such patients were randomized to receive transfusions at a hemoglobin threshold of 7 versus 9 mg/dL.  And lo! - the probability of transfusion was reduced 35%, survival increased by 4%, rebleeding decreased by 4%, and adverse events decreased by 8% in the lower threshold group - all significant!  So it is becoming increasingly clear that the data belie the sophomoric logic of transfusion.

Tuesday, December 4, 2012

The Cholesterol Hypothesis on the Beam: Dalcetrapib, PCSK9 inhibitors, and "off-target" effects of statins

The last month has witnessed the publication of three lines of research that could tip the balance of the evidence for the cholesterol hypothesis depending how things play out.  Followers of this blog know that I have a healthy degree of skepticism for the cholesterol hypothesis which was emboldened by studies of torcetrapib (blogged here and here) and anacetrapib that have come to light along with the failures of vytorin (ezetimibe; blogged here and here and hereand the addition of niacin to statins to improve cardiovascular outcomes in parallel with improvements in cholesterol numbers.

I think it's finally time to bury the CETP inhibitors. The November 29th NEJM (published online on November 5th) reports the results of the dal-OUTCOMES trial of dalcetrapib in patients with a recent acute coronary syndrome. Almost 16,000 patients were enrolled in this study of high risk patients, providing the study with ample power to detect meaningful improvements in cardiovascular outcomes - but alas, none were detected. The target is HDL, so the LDL hypothesis is not debunked by these data, but I think it is challenged nonetheless.

Bite the Bullet and Pull It: The NIKE approach to extubation.

I was very pleased to see McConville and Kress' Review article in the NEJM this week (December 6, 2012 issue) regarding weaning patients from the ventilator. I have long been a fan of the University of Chicago crew as well as their textbook and their pioneering study of sedation interruption a decade ago.

In their article, they provide a useful review of the evidence relating to the discontinuation of mechanical ventilation (aka weaning , liberation, and various other buzz words used to describe this process.) Yet at the end of the article, in describing their approach to discontinuation of mechanical ventilation, they provide a look into the crystal ball that I think and hope shows what the future may hold in this area. In a nutshell, they push the envelope and try to extubate patients as quickly as they can, ignoring inconvenient conventional parameters that may impede this approach in select instances.

Much of the research in this field has been dedicated to trying to predict the result of extubating a patient. (In the case of the most widely cited study, by Yang and Tobin, the research involves predicting the result of a predictor of the ultimate result of interest. This reminds me of Cervantes' Quijote - a story within a story within a story....but I digress.) And this is a curious state of affairs. What other endeavor do we undertake in critical care medicine where we wring our hands and so helplessly and wantonly try to predict what is going to happen? Don't we usually just do something and see what happens, making corrections along the way, in silent acknowledgment that predicting the future is often a fool's errand? What makes extubation so different? Why the preoccupation with prediction when it comes to extubation? Why not "Just Do It" and see what happens?

Wednesday, October 24, 2012

A Centrum a Day Keeps the Cancer at Bay?

Alerted as usual by the lay press to the provocative results of a non-provocative study, I read with interest the article in the October 17th JAMA by Gaziano and colleagues: Multivitamins in the Prevention of Cancer in Men. From the lay press descriptions (see: NYT summary and a less sanguine NYT article published a few days later,) I knew only that it was a positive (statistically significant) study, that the reduction in cancer observed was 8%, that a multivitamin (Centrum Silver) was used, and the study population included 14,000 male physicians.

Needless to say, in spite of a dormant hope something so simple could prevent cancer, I was skeptical. Despite decades, perhaps eons of enthusiasm for the use of vitamins, minerals, and herbal remedies, there is, to my knowledge (please, dear reader, direct me to the data if this is an omission) no credible evidence of a durable health benefit from taking such supplements in the absence of deficiency. But supplements have a lure that can beguile even the geniuses among us (see: Linus Pauling). So before I read the abstract and methods to check for the level of statistical significance, the primary endpoint, the number of endpoints, and sources of bias, I asked myself: "What is the probability that taking a simple commercially available multivitamin can prevent cancer?" and "what kind of P-value or level of statistical significance would I require to believe the result?" Indeed, if you have not yet seen the study, you can ask yourself those same questions now.

Thursday, September 27, 2012

True Believers: Faith and Reason in the Adoption of Evidence

In last week's NEJM, in an editorial response to an article demonstrating that physicians, in essence, probability adjust (a la Expected Utility Theory) the likelihood that data are true based on the funding source of a study, editor-in-Chief Jeffery M. Drazen implored the journal's readership to "believe the data." Unfortunately, he did not answer the obvious question, "which data?" A perusal of the very issue in which his editorial appears, as well as this week's journal, considered in the context of more than a decade of related research demonstrates just how ironic and ludicrous his invocation is.

This November marks the eleventh year since the publication, with great fanfare, of Van den Berghe's trial of intensive insulin therapy (IIT) in the NEJM.  That article was followed by what I have called a "premature rush to adopt the therapy" (I should have called it a stampede), creation of research agendas in multiple countries and institutions devoted to its study, amassing of reams of robust data failing to confirm the original results, and a reluctance to abandon the therapy that is rivaled in its tenacity only by the enthusiasm that drove its adoption.  In light of all the data from the last decade, I am convinced of only one thing - that it remains an open question whether control of hyperglycemia within ANY range is of benefit to patients.
Suffice it to say that the Van den Berghe data have not suffered from lack of believers - the Brunkhorst, NICE-SUGAR, and Glucontrol data have - and  it would seem that in many cases what we have is not a lack of faith so much as a lack of reason when it comes to data.  The publication of an analysis of hypoglycemia using the NICE-SUGAR database in the September 20th NEJM, and a trial in this week's NEJM involving pediatric cardiac surgery patients by by Agus et al gives researchers and clinicians yet another opportunity to apply reason and reconsider their belief in IIT and for that matter the treatment of hyperglycemia in general.

Thursday, May 24, 2012

Fever, external cooling, biological precedent, and the epistemology of medical evidence

It is rare occasion that one article allows me to review so many aspects of the epistemology of medical evidence, but alas Schortgen et al afforded me that opportunity in the May 15th issue of AJRCCM.

The issues raised by this article are so numerous that I shall make subsections for each one. The authors of this RCT sought to determine the effect of external cooling of febrile septic patients on vasopressor requirements and mortality. Their conclusion was that "fever control using external cooling was safe and decreased vasopressor requirements and early mortality in septic shock." Let's explore the article and the issues it raises and see if this conclusion seems justified and how this study fits into current ICU practice.


These are related but distinct issues that are best considered both before a study is planned, and before its report is read. A clinical trial is in essence a diagnostic test of a hypothesis, and like a diagnostic test, its influence on what we already know depends not only on the characteristics of the test (sensitivity and specificity in a diagnostic test; alpha and power in the case of a clinical trial) but also on the strength of our prior beliefs. To quote Sagan [again], "extraordinary claims require extraordinary evidence." I like analogies of extremes: no trial result is sufficient to convince the skeptical observer that orange juice reduces mortality in sepsis by 30%; and no evidence, however cogently presented, is sufficient to convince him that the sun will not rise tomorrow. So when we read the title of this or any other study, we should pause to ask: What is my prior belief that external cooling will reduce mortality in septic shock? That it will reduce vasopressor requirements?

Sunday, December 18, 2011

Modern Day Bloodletting: The Good Samaritan, the Red Cross, and the Jehovah's Witness

How many studies do you suppose that we need before doctors realize that their tendency to want to transfuse blood in every manner of patient admitted to the hospital is nothing more than an exercise in stupidity futility based on the normalization heuristic?  It's a compelling logic and an irresistible practice, I know.  The hemoglobin level is low, that can't be good for the heart, circulation, perfusion, oxygen delivery, you name it.  If we just give a transfusion or two, everything will be all better.  I can hear family members on their mobile phones reassuring other loved ones that the doctors are acting with great prudence and diligence taking care of Mr. Jones, having perspicaciously measured his hemoglobin (as by routine, for a hospital charge of ~$300/day - the leviathan bill and the confused, incredulous faces come months later - "why does it cost so much?"), discovered that perilous anemia, and ordered two units of life-saving blood to be transfused.  It's so simple but so miraculous!  Thank God for the Red Cross!

Not so fast.  The TRICC trial published in 1999 demonstrated that at least in critically ill patients, using a lower as compared to a higher transfusion threshold led to a statistically insignificant trend towards improved outcomes in the lower threshold group.  That is, less blood is better.  For every reason you can think of that transfusion can improve physiological parameters or outcomes, there is a counterargument about how transfusions can wreak havoc on homeostasis and the immune system (see :Marik_2008_CCM, and others.)

Not to mention the cost.  My time honored estimate of the cost of one unit of PRBCs was about $400.  It may indeed be three times higher.  That's right, $1200 per unit transfused, and for reasons of parity or some other nonsense, in clinical practice they're usually transfused in "twos".  Yep, $2400 a pair.  (Even though Samaritans donate for free, the cost of processing, testing, storage, transportation, etc. drive up the price.)  What value do we get for this expense?

Thursday, November 10, 2011

Post-hOckham analyses - the simplest explanation is that it just plain didn't flipp'n work

You're probably familiar with that Franciscan friar Sir William of Ockham, and his sacred saw. Apparently the principle has been as oversimplified as it has ignored, as a search of Wikipedia will attest. Suffice it to say, nonetheless, that this maxim guides us to select the simplest from among multiple explanations for any phenomenon - and this intuitively makes sense, because there are infinite and infinitely complex possible explanations for any phenomenon.

So I'm always amused and sometimes astonished when medical scientists reappraise their theories after they've been defeated by their very own data and begin to formulate increasingly complex explanations and apologies, so smitten and beholden to them as they are. "True Believers" is what Jon Abrams, MD, one of my former attendings, used to call them. The transition from scientist to theist is an insidious and subversive one.

The question is begged: did we design such and such clinical trial to test the null hypothesis or not? If some post-hoc subgroup is going to do better with therapy XYZ, why didn't we identify that a priori? Why didn't we test just THAT group? Why didn't we say, in advance, "if this trial fails to show efficacy, it will be because we should have limited it to this or that subgroup. And if it fails, we will follow up with a trial of this or that subgroup."