In this section
The Irreproducibility Problem Affecting 50% of Today’s Published Studies
According to Science20.com, a recent study has revealed that over 50% of the biomedical research published today cannot be successfully reproduced. One 2005 study suggested that more than half of all studies were irreproducible or otherwise flawed. Why does the irreproducibility problem exist? And how can it be solved?
This issue has lasting knock-on effects. For one thing, the inability to reproduce studies means that expensive follow-up work often turns out to be a dead end. For another, studies aren’t usually marked or highlighted in any way as irreproducible even after the problem has been discovered. There’s no suggestion that irreproducible studies be retracted, but allowing them to remain unchanged in the public domain contributes to the spread of inaccurate information that plagues the world of biomedical science.
There are a few reasons behind the growing irreproducibility problem. The main ones come down to human error – either mistakes are made during the study itself or errors occur when it is being written up. These mistakes don’t have to be colossal or even noticeable; as Science 20 points out, documentation errors like the missing of steps when recording methodology are enough to render a study irreproducible.
Based on these figures it’s tempting to jump to the idea that the world of biomedical research is fraught with fraud, but that isn’t the case. There are over 400 retractions in the biomedical and life science research fields every year, and only around 43% of them are down to fraud.
Steps are being taken to solve the problem. One of the most notable solutions is the founding of the Reproducibility Initiative. This collaboration between Science Exchange, PLOS, Figshare, and Mendeley offers researchers the opportunity to submit experiments to a verified lab that will reproduce the study. If the results can be verified, the submitter is licensed to use the Reproducibility Initiative’s “Independently Validated” badge as well as having their work published in a validation study.
Another way of addressing the issue is offered by the ongoing improvements to data collection in the in vivo field. ActualHCA offers a way to store copious amounts of data from 24/7 behaviour monitoring. This is easily accessible and auditable after the fact, and any unusual behaviour from rodent test subjects can be tracked down and reviewed quickly and easily. Equipment and software like ActualHCA take the unpredictable factor of human error out of the data recording process.
It’s also significant that the data produced using ActualHCA is richer than that offered by standard testing methods. The more detail and nuance there is available, the easier it is to reproduce a study.