In this section
The Latest Guidelines for Increasing Reproducibility
The issue of reproducibility in biomedical science has been thoroughly discussed in recent months – we recently covered the topic ourselves. In April, the Academy of Medical Sciences joined with the Wellcome Trust, the Biotechnology and Biological Sciences Research Council (BBSRC), and the Medical Research Council (MRC) to hold a symposium on the issue. A report from this symposium has now been published.
The existence of irreproducibility itself is not an issue; as noted in the report, there are numerous fields in which a certain lack of reproducibility is expected. However, the scale of the issue across a range of fields is a severe concern. In psychology, for example, a recent wide-scale reproducibility attempt revealed that almost two thirds of study results from recent years couldn’t be reproduced.
As the report points out: “there is no single cause of irreproducibility”. However, there are what it terms “examples of poor practice”. These are:
• Data dredging
• Omitting null results
• Underpowered study
• Underspecified methods
• Weak experimental design
The report lists a few strategies that could help to resolve the issue. These are listed as:
• Open data: the open sharing in the field of “results and the underlying data”
• Pre-registration of study protocol
• Automation: technologically-standardised research practice reduces human error and ensures everyone is using the same methods to achieve the same goals
• Open methods: similar to the “open data” goal, only with study protocol
• Post-publication review
• Reporting guidelines
A Nature interview with Professor Dorothy Bishop, who led the April meeting, covered some of the other possible solutions mentioned in the report. Professor Bishop stated that the issue of irreproducibility has many causes aside from what happens before, after, and during studies themselves. She suggests two approaches: first, a “top-down” approach, where “funders, institutions, and journals… change practices to favour reproducible research, rather than sexy research”. This sums up a pervasive issue: scientists who spend many years working patiently and thoroughly lose recognition because they haven’t published in a while. When they do publish, the results aren’t especially startling, so the attention goes to exciting, but often irreproducible, results instead.
The second approach that Professor Bishop describes is a “bottom-up” approach. Put simply, she states that researchers should be trained and educated in best practice by their superiors. As the report points out, this education should start from the beginning of careers in the sciences and persist to the end.
ActualHCA can play an important role in improving reproducibility. It automatically records and documents behaviour, eliminating issues like human error and differences in opinion, and enables the reliable and consistent output of rich, statistically robust data.
To read the symposium report from The Academy of Medical Sciences, click here. To read the interview with Professor Bishop in Nature, click here. For more information on ActualHCA, visit our product page or get in touch today.