Scientists looking to verify their research results now have an option besides redoing everything themselves. The start-up Science Exchange, along with open-access publisher Public Library of Science (PLoS) and open data repository Figshare, have launched the Reproducibility Initiative, a program to help scientific researchers validate their findings.
"We found that industry, in particular, was starting to use our platform to do independent replications of interesting results that they were interested in licensing from academic labs," she said. "Because they were so concerned about this issue of lack of reproducibility, they were looking for a way to get the results independently reproduced in a cost-effective way."
To respond to this need—not only for industry groups but for academic labs as well as funding agencies—Science Exchange brought on an advisory board of scientists interested in the issue of reproducibility. When a research group submits their study to the Reproducibility Initiative, the board will match the study with someone from the existing network of experimental service providers. This provider, which remains anonymous to the original researchers, receives a fee to replicate the original experiments as closely as possible and submit their results to the initiative.
If their studies are validated, the researchers receive a "certificate of reproducibility" and the option to upload their primary data to Figshare. They also can publish their validation in a special upcoming issue of the journal PLoS ONE devoted to reproducibility. If the studies are not validated, the researchers still have the option of publishing in PLoS ONE but are not required to do so. "We don't want to put punitive pressure on people who want to take part," said Iorns.
The cost of validation—the outside facility's fees plus a 5% cut paid to Science Exchange—will likely be about 10% of the price of the original experiments, estimated Iorns, and will be covered by the researchers' own grants. But that’s something she hopes to change. "We're working really hard with funding agencies to get them to fund not only original exploratory studies but also to fund validations," she said. "But we haven't gotten that pinned down yet."
In light of several recent studies showing that attempts to reproduce laboratory studies can be astonishingly unsuccessful, "it really makes sense for the funding agencies to pay," said Iorns. "If so much of the research that they're funding is not reproducible, that's a huge waste of money."
As for individual researchers, Iorns hopes that they'll be motivated by potential rewards of an additional publication and the certificate verifying reproducibility, which "will be looked on favorably by industry partners and by future funders."