In 2005, Tao Huang and colleagues from the Swedish University of Agricultural Sciences (SUAS) and the French National Institute for Agricultural Research published a paper in Science examining a mystery of plant biology: what controls the seasonal development of flowers in plants?
Until this point, plant biologists did not understand the molecular process by which plants translated daylight information received by the leaves into signals to turn genes on at the site of budding flowers (1). But Huang and his colleagues appeared to have found the answer. Based on data from real-time quantitative PCR (qPCR) experiments, the team concluded that messenger RNA (mRNA) was responsible for taking the instructions for flowers to start blooming from the leaf to the growth site in Arabidopsis.
The paper attracted much attention, being named as a “Breakthrough of the Year 2005” runner-up by Science (2) and as one of the top “Signaling Breakthroughs of the Year” by Science Signaling (3). A review published in the Journal of Integrative Plant Biology also described the paper as a “major scientific breakthrough in 2005.” (4)
There was only one small problem: the qPCR data was flawed. When Tao Huang left the Swedish University of Agricultural Sciences, his colleagues became concerned when they compared the original data with the published qPCR data. Data points were removed. Others were increased in weight in the analysis. When his colleagues re-analyzed the original qPCR data, the significant results vanished. Huang’s colleagues then attempted to reproduce the findings of the article, but could not. In the, end the authors had no choice but to retract the paper.
The Huang story is not unique. During the past decade, several high-profile cases of faulty research have been linked to inconsistent qPCR techniques and experiments. Stephen Bustin, a molecular science professor at the school of medicine and dentistry at Queen Mary University of London, is working to prevent such instances from happening in the future.
“People are beginning to appreciate and understand that something has to be done about the quality of the PCR assays that are being published,” Bustin said in an interview with BioTechniques, “because we are all suffering [from the effects] of poor results and irreproducible data.”
Scientists are constantly modifying the technique and developing a variety of different combinations of assays, protocols, primers, and probes, to overcome the limitations of standardized qPCR methods. The plethora of techniques and materials makes it difficult for independent researchers to corroborate results of published experiments.
So in April 2009, Bustin and an international team of nine scientists joined forces and developed a set of guidelines for the publishing qPCR results. The guidelines were meant to make the rapidly evolving qPCR field less confusing for those involved, according to team member Carl Wittwer, a professor of pathology and biomedical engineering at the University of Utah.
“There are so many different ways to perform qPCR that [guidelines] might help authors perform better experiments,” said Wittwer. “It might help reviewers and editors of journals, if they had some sort of guideline from which to judge the technical or scientific quality of qPCR experiments; and finally, it might help readers who consume the scientific literature and often want to repeat and verify the results of qPCR experiments.”
The resulting minimum information for publication of quantitative real-time PCR experiments (MIQE) guidelines includes about 60 guidelines for the publication of qPCR experiments and a detailed checklist for researchers so that critical steps and materials used in performing their experiments are included with their publication submissions (5).
One year later
Since the MIQE guidelines were published last year, the scientific community has had a mixed reaction, according to Bustin. He and some of his colleagues have been on two lecture tours to introduce the MIQE guidelines to researchers.
Bustin and Wittwer agree that many scientists have voiced their support for the guidelines, which is somewhat of a surprise. “Researchers are supposed to be highly independent and sort of determine their own fate and future and the way they do things,” said Wittwer, “so I expected more resistance,” he explained.
Still, there is plenty of dissent. Wittwer said that this is because the guidelines are complicated and expensive to implement. “There are a lot of experimental requirements in there,” he said. “If you are a limited resource academic having to make decisions on the extent of experimentation you do, it’s a lot easier to try to do something quick and dirty that’s probably right, than to do a complete validated, verified study that follows guidelines that are by their nature, strict.”
Some researchers whose projects are funded by biotech companies that produce qPCR materials may feel pressured to exclude product information that would reveal a possible conflict, according to Wittwer. And because some of these companies are reluctant to include detailed product information—such as probe and primer sequences—researchers who use these materials can’t include these details in their reports.
To overcome this obstacle, Bustin has brought his message to the life science industry, targeting manufacturers of qPCR supplies. “The response from companies is overwhelmingly positive,” he said.
A major corporate supporter of the MIQE guidelines has been Bio-Rad, who even sponsored Bustin’s lecture tours to promote the guidelines. The company is trying to make their customers more MIQE-aware and are providing qPCR analysis software that conforms to these guidelines. Bio-Rad is also making some internal adjustments; the company is training their employees to implement the guidelines in future publications. “It is greatly to the credit of companies such as Bio-Rad that they are prepared to invest in this educational exercise,” Bustin said.
Another company on board with the MIQE guidelines is Thermo Fisher Scientific, Inc. To help researchers comply with the guidelines, the company is now including all probe and primer sequence information for some of its Solaris qPCR assays. This sort of information is not included with many commercial assays, though Bustin and Wittwer hope that other companies will follow similar suit with their own qPCR products.
But despite the zeal of some scientists and companies in adopting the guidelines, Bustin believes that the watershed mark lies with journals themselves. “As an editor, surely your job is to lead people. If you know that people are producing data that isn’t very good, then you should start implementing a few policy guidelines,” said Bustin. “They don’t have to be the MIQE guidelines, but there has to be certainly some quality issues or quality standards that should be implemented.”
Several scientific journals are climbing aboard the MIQE bandwagon, according to Bustin. “BMC journals are particularly receptive to MIQE, and the journal I am associated with, BMC Molecular Biology, is actively discussing how best to address the problem of ensuring that qPCR data are reliable,” he said.
But high-impact journals like Nature, Nature Medicine, and Science are another story. They continue to publish qPCR data that is “uniformly appalling,” according to Bustin. He says that these journals refuse to require detailed research reports from scientists. Because of this, Bustin said that researchers are less likely to turn them in of their own volition, thereby fueling the cycle of faulty research.
In an April 2010 article, Nature Medicine editor-in-chief Juan Carlos Lopez explained the journal’s position on the guidelines. “We would be delighted to embrace the [MIQE] guidelines, but we are not really sure that the guidelines are accepted by the scientific community,” he said.
Bustin disagrees. He does not believe that widespread acceptance must occur before journals begin implementing some form of guidelines when it comes to the publication of data. “None of this is meant to de dogmatic. It is simply, in my opinion, a very commonsensical approach to how we get good assays,” said Bustin.
Guiding the future
At the moment, Bustin and his colleagues are considering adding to the guidelines based on the responses they have received from companies, journals, and researchers. “Talking with colleagues, you get reality tests of what people find very difficult to do and what may be more reasonable to do,” said Wittwer.
The group is considering developing guidelines for multiplex assays and is debating the requirement of probe/primer sequences for qPCR. The current guidelines require submission of primer sequences but not probe sequences because many commercial assays currently don’t include the information. For his part, Bustin feels that the sequences should be reported.
“In order to be able to evaluate the quality and reliability of an assay—and if I want to reproduce it—then I would need to know the probe sequence,” he said. “I think in terms of scientific integrity, it’s important to have a complete assay published.”
To promote the guidelines to scientists and biotechnology companies, Bustin plans to tour several countries over the next year. He will speak at a workshop in St. Louis, MO sponsored by Sigma-Aldrich, as well as several scientific conferences in Germany. He will also attend Bio-Rad–sponsored scientific meetings in Singapore, Australia, and China.
For his part, Wittwer will also promote the guidelines at the scientific meetings he attends. He hopes to convince more science journal editors that the guidelines are important and useful tools to improve the quality of qPCR-based research.
“Companies, journals, and—most importantly—the primary investigators are crucial to that process,” he said. “Perhaps the guidelines have a contribution to helping that happen as qPCR evolves.”
References
1. Huang, T., H. Böhlenius, S. Eriksson, F. Parcy, O. Nilsson. 2005. The mRNA of the Arabidopsis gene FT moves from leaf to shoot apex and induces flowering. Science 309:1694-1696.
2. 2005. Breakthrough of the year, the runners-up. Science 310:1880 – 1885
3. Adler, E.M., N.R. Gough, L.B. Ray. 2006. 2005: signaling breakthroughs of the year. Sci. STKE, 316: p. eg1.
4. Yu X, J. Klejnot, C. Lin. 2006. Florigen: one found, more to follow? J. Integr. plant biol. 48:617-621
5. Bustin, A., V. Benes, J. Garson, J. Hellemans, J. Huggett, M. Kubista, R. Mueller, T. Nolan, et al. 2009. The MIQE guidelines: Minimum information for publication of quantitative real-time PCR experiments. Clin. Chem. 55:611-622
