We're so good at medical studies that...
aaCharley
26 Mar 2010
We're so good at medical studies that most of them are wrong
By John Timmer
It's possible to get the mental equivalent of whiplash from the latest medical findings, as risk factors are identified one year and exonerated the next. According to a panel at the American Association for the Advancement of Science, this isn't a failure of medical research; it's a failure of statistics, and one that is becoming more common in fields ranging from genomics to astronomy. The problem is that our statistical tools for evaluating the probability of error haven't kept pace with our own successes, in the form of our ability to obtain massive data sets and perform multiple tests on them. Even given a low tolerance for error, the sheer number of tests performed ensures that some of them will produce erroneous results at random.
Pretty worthwhile read as it highlights the shortcommings of the studies that we have come to rely on. More along the lines of why to be cautious when looking at the results, even when there is not a particular flaw in the study.
By John Timmer
It's possible to get the mental equivalent of whiplash from the latest medical findings, as risk factors are identified one year and exonerated the next. According to a panel at the American Association for the Advancement of Science, this isn't a failure of medical research; it's a failure of statistics, and one that is becoming more common in fields ranging from genomics to astronomy. The problem is that our statistical tools for evaluating the probability of error haven't kept pace with our own successes, in the form of our ability to obtain massive data sets and perform multiple tests on them. Even given a low tolerance for error, the sheer number of tests performed ensures that some of them will produce erroneous results at random.
Pretty worthwhile read as it highlights the shortcommings of the studies that we have come to rely on. More along the lines of why to be cautious when looking at the results, even when there is not a particular flaw in the study.
aaCharley
26 Mar 2010
Statistics must be the NEW hot topic for researchers. Another paper discussed in ScienceNews:
Odds are, Its Wrong
Be careful out there.
Odds are, Its Wrong
Be careful out there.
