to BioTechniques free email alert service to receive content updates.
GUIDING OUR PCR EXPERIMENTS
 
Jeffrey Perkel, Ph.D.
BioTechniques, Vol. 58, No. 5, May 2015, pp. 217–221
Full Text (PDF)
Abstract

The MIQE guidelines for qPCR and dPCR have been around for a while now, but few are taking advantage of this resource. Jeffrey Perkel looks at challenge of standardizing PCR.

When Mary Alikian, a clinical scientist and “part-time PhD student” at Imperial College London, began her doctoral research, she decided to use digital PCR (dPCR). Alikian, who was interested in investigating RNA biomarkers associated with chronic myelogenous leukemia, quickly realized she had a problem though: no one in the lab had experience with dPCR, so no one could teach her how to do it correctly. “I had no clue what things I should consider and what things I should not.”



If she were using traditional PCR, it might not have mattered so much, as PCR can be fairly forgiving. But dPCR, much like real-time PCR (qPCR), is a quantitative methodology that is considerably more complicated. The method involves dedicated instrumentation, protocols, and variables that are not used in the traditional PCR assay, but the resulting data can be used to identify everything from dysregulated genes to chromosomal variation. Improperly trained researchers using dPCR to quantify, say, mRNA abundance, will inevitably obtain a number, but whether that number accurately represents the sample itself is less certain.

Digital guide

Fortunately for Alikian, a group of reproducibility-focused researchers had recently developed a kind of roadmap for dPCR experiments, called the Minimum Information for Publication of Quantitative Digital PCR Experiments (digital MIQE) guidelines. The concept behind digital MIQE was to guide researchers through the “hows” and “whys” of dPCR, including experimental design and analysis. The guidelines even offer specific recommendations on how to report dPCR experiments for maximum transparency. Jim Huggett, Principal Scientist for Nucleic Acid Research at the international life sciences company LGC, who was lead author on the digital MIQE paper, calls the guidelines “a call for information.” Digital MIQE, he explains, “is about including in your publication the information that makes it possible for me to reproduce your work.”

For her part, Alikian viewed digital MIQE as a set of best practices for producing high-quality, trustworthy data. “You know that you are doing the best, to your knowledge,” she says, “because experts are providing you with guidance of what things you need to consider.”

Yet oddly among the growing number of researchers who are using dPCR in their experiments, Alikian is in the minority. Since publication in 2013, the digital MIQE guidelines have been cited 74 times, according to Google Scholar. PubMed lists over 400 dPCR papers in the same period. Similarly, the original MIQE guidelines, digital MIQE's qPCR-focused predecessors, have been cited some 3200 times since publication in 2009, although more than 26,000 qPCR publications have appeared over the same time period.

Stephen Bustin, Professor of Molecular Medicine at Anglia Ruskin University in the UK, who was corresponding author on the original MIQE guidlines and senior author on the digital MIQE paper, has spent years lecturing about and documenting the problems of qPCR and dPCR transparency and reproducibility. In 2014, he and his colleagues reported that of 179 papers published between January 2006 and August 2013 using qPCR in colorectal cancer biomarker studies, only 8% used more than a single reference gene (as recommended in the MIQE guidelines). Only 13% of the studies indicated if their reference gene choices were validated, and 70% used 1 of only 3 reference genes (ACTB, GAPDH, or 18S RNA), suggesting they were never validated (1).

Adoption issues

In an editorial in the journal Biomolecular Detection and Quantification, a publication Bustin co-founded specifically to promote high-quality quantitative studies, he reported that of 10 articles “selected at random” with RT-qPCR data published by journals in the Nature Publishing Group in 2014, none reported such key MIQE details as RNA integrity, RNA purity, reverse transcriptase conditions, or PCR efficiency, and all used just a single unvalidated reference gene for transcript normalization, despite evidence suggesting the inaccuracy of that approach (2).



“It's a litany,” Bustin says. “Wherever you look, there's a problem.”



And according to Bustin, the problem is real. Using the MIQE guidelines, per se, might not be the key to solving every issue of scientific reproducibility, but for qPCR and dPCR, the steps these articles lay out can help researchers ensure their data are well documented and rigorous, and thus more likely to be accurate. Conversely, ignoring the guidelines can yield data of lower reliability and robustness, thus wasting researcher time and resources—which is especially galling in light of contracting research budgets. “The conclusion from other people,” he says, citing figures published in The Lancet(2), “is that approximately 85% of research funding is wasted.”

Why would researchers choose to publish poor-quality data? To some extent, many may not know better. Andrzej Pietrzykowski, Assistant Professor in the Department of Animal Sciences at Rutgers University and a Visiting Scientist at the Martinos Center for Biomedical Imaging at Massachusetts General Hospital, who follows the dPCR MIQE guidelines in his own work, says many researchers are unaware of MIQE, and those who are aware say that following the guidelines set forth in MIQE requires extra work, and thus, more money. Francisco Bizouarn, Global Digital Applications Specialist at Bio-Rad Laboratories, also suspects that researchers may find the MIQE and digital MIQE checklists, with 80-plus items each, “intimidating.”

Finding common ground

Of course, not every reaction requires that every box be checked. The MIQE guidelines strive for transparency, not constraint, and thus allow for common sense, especially when performing large-scale screens of multiple targets. Still, says Bustin, researchers following the qPCR MIQR guidelines need at the very least to test samples for the presence of PCR inhibitors, measure PCR efficiency, and screen gene panels to identify suitable genes for normalization. “[For] everything else, you just record what you're doing, but these three things require significant additional effort,” he says. Similarly, the digital MIQE guidelines require users to measure such values as the number of digital PCR partitions, the average number of DNA molecules per partition, their volume, and the variance in that volume—numbers that help researchers and reviewers better understand the quality and reliability of a reaction.



What's really needed to broaden MIQE compliance, Bustin says, is the engagement of journal editors. Yet despite the flurry of editorials and peer-reviewed research over the past few years on the problem of data reproducibility, Bustin and his colleagues have found that MIQE adherence actually correlates inversely with journal impact factor (although that may be in part because papers in higher impact journals tend to combine qPCR and dPCR with other types of data, providing a kind of cross-validation, Bizouarn suggests). Nature Methods and BioTechniques make no mention of MIQE in their author guidelines, but PeerJ and Nucleic Acids Research do. Clinical Chemistry actually requires authors to complete and submit a MIQE checklist along with their manuscript.

Interestingly, says Bustin, one group that has been particularly supportive is instrumentation vendors. Though there's no such thing as a “MIQE-certified” instrument or kit, many vendors actively promote the guidelines to their customers. Bio-Rad, for instance, sponsors a mobile app, available for both iOS and Android, that quantifies MIQE compliance in real time. Afif Abdel Nour, an applications scientist at Bio-Rad who co-developed the app during his former position at the Institut Polytechnique LaSalle Beauvais, says it provides “a digital checklist.” Users can save the state of a project and export it for submission to a journal or to share with colleagues—and manuscript reviewers can use the app to ensure articles are up to snuff. The app has been downloaded some 11,000 times for iOS and 3000 times for Android, according to Abdel Nour.

Such tools may make researchers more aware of good PCR practices. But Huggett suggests MIQE may also face a perception problem. Some researchers, he says, believe the guidelines are “a dictatorship so to speak, or a police state.” Yet MIQE, he notes, is not a set of rules but suggestions. Rather than forcing researchers to pile on extra controls, the goal is to help them make the most of what they have in order to make every reaction count. And the only way to do that, he says, is by recognizing the limitations of the method and the data it produces. “That to me is the key behind MIQE. It's about providing that information and making conclusions that are appropriate.”

References
1.) Dijkstra, J.R., L.C. van Kempen, I.D. Nagtegaal, and S.A. Bustin. 2014. Critical appraisal of quantitative PCR results in colorectal cancer research: Can we rely on published qPCR results?. Mol Oncol. 8:813-818.

2.) Bustin, S.A. 2015. The reproducibility of biomedical research: Sleepers awake!. Biomol. Detect. Quantif. 2:35-42.