Tuesday, April 1, 2014

Absolute Confusion: How Researchers Mislead the Public with Relative Risk

This article in Sunday's New York Times about gauging the risk of autism highlights an important confusion in the appraisal of evidence from clinical trials and epidemiological studies that appears to be shared by laypersons, researchers, and practitioners alike:  we focus on relative risks when we should be concerned with absolute risks.

The rational decision maker, when evaluating a risk or a benefit, is concerned with the absolute magnitude of that risk or benefit.  A proportional change from an arbitrary baseline (a relative risk) is irrelevant.  Here's an example that should bring this into keen focus.

If you are shopping and you find a 50% off sale, that's a great sale.  Unless you're shopping for socks.  At $0.99 a pair, you save $0.50 with that massive discount.  Alternatively, if you come across a 3% sale, but it's at the Audi dealership, that paltry discount can save you $900 on a $30,000 Audi A4.   Which discount should you spend the day pursuing?  The discount rate mathematically obscures the value of the savings.  If we framed the problem in terms of absolute savings, we would be better consumers.  But retailers know that saying "50% OFF!" attracts more attention than "$0.50 OFF!" in the sock department.  Likewise, car salesmen know that writing "$1000 BELOW INVOICE!" on the windshield looks a lot more attractive than "3% BELOW INVOICE!"



The rational shopper makes no such distinctions - s/he is looking for big absolute savings rather than big proportional discounts.  If s/he has limited time and is going to invest it in the goal of saving the most money, s/he will pursue opportunities with large absolute savings, rather than large proportional discounts.  But there are abundant data from behavioral economics that show that we are not rational shoppers - to our peril.

Likewise with medical and epidemiological data.  If you have cancer, and there is a chemotherapy drug that cuts your absolute risk of dying by 10%, it matters little whether it reduces it from 50% to 40% (with a relative risk reduction of 20%) or whether it reduces it from 20% to 10% (with a relative risk reduction of 50%) - 10% is 10% (just like $10 is $10), regardless of the baseline from which we start.  Note also that as we approach zero risk, small absolute risk reductions can produce large relative risk reductions.  Conversely, non-trivial absolute risk reductions on the order of 5% can appear piddling when the baseline risk is 75%.

Similarly, relative risks can scare us more than there is cause to be scared, as in the case of autism.  The risk of Down Syndrome in offspring increases by five-fold as maternal age increases from 30 to 40 years.  A 500% increase!  A relative risk of 5!  But the absolute risk goes from 0.0005 to 0.0025 - an absolute risk increase of 0.002, or two tenths of a percent.  When most people (me included) think of 0.2%, we don't get too worked up.  (Maybe we should.)

So why do we continue to promulgate relative risks?  Here are some ideas:

  1. Like the sock retailer and the car salesman, researchers (especially those employed by for-profit pharmaceutical companies) want to promote their findings and their wares.  It helps get grants, leads to notoriety, increases press coverage, sells drugs, and enhances profits.  So if the relative number is greater than the absolute number (and it always is), that's the one they're going to report.  Even if it inflates a minuscule benefit because of a low baseline risk rate.
  2. In some epidemiological studies (e.g. case control, meta-analysis) an absolute risk reduction is not possible to ascertain and odds ratios (a surrogate for relative risk) are the only way to report the results.
  3. If you want to compare the "prowess" of therapies across diseases with different outcome rates, (perhaps as a surrogate for the prowess of the researchers who investigate them), it seems more fair to compare relative risk reductions than absolute risk reductions.  It's like a weight loss competition - if the guy who enters at a weight of 495 pounds loses 20 pounds, is he really a superior competitor to the guy who entered at a weight of 225 pounds and lost 15 pounds?  Insomuch as the competition for academic prowess resembles a weight loss contest, comparing relative reductions might be the way to go.
  4. If you're a public health type, and you want to convince somebody to engage in some behavior which on the population level has measurable benefits, you have to convince people on the individual level to engage in behaviors (motorcycle helmets, seat belts, statins, etc.) which only very slightly decrease their individual risk.  Relative risks are a way to do that, because they "appear" greater and more convincing than absolute risks.
The problem is that relative risks don't tell us how much benefit we get from a therapy or the value of an epidemiological finding - they only tell us that benefit in reference to a baseline which we may or may not know.  And if we need to know the baseline in order to determine the absolute value of the benefit (by mathematical deconstruction), then we should just report the absolute benefit in the first place.

1 comment:

  1. Many reports in the media about the benefits of treatments present risk results as relative risk reductions rather than absolute risk reductions. This often makes the treatments seem better than they actually are.

    ReplyDelete