Efficiency clustering for low-density microarrays and its application to QPCR
© Lock et al; licensee BioMed Central Ltd. 2010
Received: 31 March 2010
Accepted: 20 July 2010
Published: 20 July 2010
Pathway-targeted or low-density arrays are used more and more frequently in biomedical research, particularly those arrays that are based on quantitative real-time PCR. Typical QPCR arrays contain 96-1024 primer pairs or probes, and they bring with it the promise of being able to reliably measure differences in target levels without the need to establish absolute standard curves for each and every target. To achieve reliable quantification all primer pairs or array probes must perform with the same efficiency.
Our results indicate that QPCR primer-pairs differ significantly both in reliability and efficiency. They can only be used in an array format if the raw data (so called CT values for real-time QPCR) are transformed to take these differences into account. We developed a novel method to obtain efficiency-adjusted CT values. We introduce transformed confidence intervals as a novel measure to identify unreliable primers. We introduce a robust clustering algorithm to combine efficiencies of groups of probes, and our results indicate that using n < 10 cluster-based mean efficiencies is comparable to using individually determined efficiency adjustments for each primer pair (N = 96-1024).
Careful estimation of primer efficiency is necessary to avoid significant measurement inaccuracies. Transformed confidence intervals are a novel method to assess and interprete the reliability of an efficiency estimate in a high throughput format. Efficiency clustering as developed here serves as a compromise between the imprecision in assuming uniform efficiency, and the computational complexity and danger of over-fitting when using individually determined efficiencies.
Array and microarray based methods are the mainstay of molecular biology. Recently, lower-density or targeted arrays have been introduced. These comprise on the order of 10-1000 targets and represent an intermediate between 1 target assays, e.g. the viral load assay for HIV, and genomewide microarrays, comprising 10,000 -1,000,000 targets, e.g. Affymetrix™ arrays. Analytically they can be treated as either a collection of individual assays or as microarrays.
We are interested in discovering novel statistical approaches to the analysis of these intermediate density arrays. On the one hand high density microarray-based analysis methods do not capture all the information that is available. This results in lower than possible linear range and lower discriminating power. On the other hand, applying analysis methods developed for a single assay to arrays is overly complex and likely to introduce inacurracies due to overfitting problems.
Practically lower density arrays are based on real-time quantitative polymerase chain reaction (QPCR). Real-time QPCR measures the amount of product at each cycle of the reaction either by binding of a uorescent, double strand-specific dye (SYBR™ green) or by hybridization to a third sequence-specific, dual-labeled uorogenic oligonucleotide probe (molecular Beacon, TaqMan™ ). These have been used very successfully to profile messenger RNAs and microRNA [1–5]. Other assay formats use capture oligonucleotides or other means (e.g. Nanostring™ or Luminex™).
The simplest way to compare relative expression ratios is the so-called method, which assumes (i) uniform and (ii) perfect efficiency: the amount of product doubles exactly after each PCR cycle for each primer in the array. However, most QPCR reactions do not attain optimal efficiency [6, 7]. Even small deviations from an assumed efficiency level can lead to multiple-fold inaccuracies in expression measurements . Therefore, differing efficiencies among primer-pairs, as well as the reliability of the efficiency estimates must be considered.
The most accurate and also most complex method is to calculate individual primer pair efficiencies or to use an absolute standard curve. This method does not assume uniform efficiency among different primer pairs and it does not assume perfect efficiency. The well known REST method  is an example of this strategy. However, this approach becomes impractical and extremely costly to use even for low density arrays.
More important, using individually adjusted efficiencies brings about the problem of overfitting and it requires that the individual efficiencies were determined with equal confidence. For instance, a 96 primer targeted QPCR array, would require the computation of 962 individual efficiency ratios. These need to be tracked throughout the analysis process and each ratio is associated with its own error, which also needs to be propagated throughout the analysis process.
The methods presented here use the serial dilution method  to obtain initial primer pair efficiencies. We expand on this method, introducing transformed normality-based confidence intervals as a novel tool to interpret the reliability of an efficiency estimate. We propose a rule to identify unreliable primers, as well as a robust algorithm to cluster primers with similar efficiencies. Finally, we show how differences in efficiency can be applied for more accurate comparisons of relative gene expression.
The primer arrays and experimental methods were previously described [11, 12]. Primers were from commercial sources (MWG Inc.) and resuspended at 100 pmol per microliter in 0.1x Tris-EDTA ph 8.0. The QPCR reaction contained 2.5 microliter of primer mix at a starting concentration of 300 nM combined with 7.5 microliter SYBR Green 2× PCR mix (Applied Biosystems, Inc.) and 5 microliter target DNA. It was subjected to real-time QPCR on an Opticon2 cycler (MJR Inc.) using standard cycling conditions . This particular QPCR array was directed against every open reading frame of human herpesvirus 8. This particular virus is made of linear double-stranded DNA. Thus we were able to use the same purified viral DNA as a common target for all primers.
The R programming environment for statistical computing and graphics  was used for all computation and statistical analysis. Stand-alone R functions were developed to automate the analysis for a given serial dilution table. Functions were created to provide individual efficiency estimates with confidence, identify and remove unreliable primers, cluster amplification efficiencies, and adjust C T values in a table based on user-defined efficiencies. All functions were included in the R script Primer Efficiency Analysis (Additional file 1 - PEA.r); a reference manual is available with setup instructions, detailed function descriptions and illustrative examples (Additional file 2 - PEA User's Guide).
Results and Discussion
Estimating Efficiency with Confidence
The number of cycles required to reach a certain level of product (C T ) is measured for 96 primers, each with initial dilution levels 1, 0.1, 0.01, and 0.001. C T values above 40 indicate undetectable product and are considered to be missing values (NA) for subsequent calculations. If the dilution curve for a primer contains 2 or more missing values (50% of total), statistical analysis for a linear fit is impossible and the primer is removed from the analysis. Of the 96 original primers in our dataset, 3 were removed due to missing values.
Before continuing, we remove those primers for which a linear fit is not statistically significant, based on the t-statistic for the estimated slope coefficient . If the corresponding p-value is above 0.05, the primer is considered unreliable and removed from further analysis. Using the t-statistic rather then the regression coefficient R2 initially serves the following purpose. The t-statistic accounts for the number of dilution steps, whereas a threshhold based on the coefficient of determination R2 does not (R2 = 1 for 2 data points and likely decreases with the addition of more measurements). More dilution steps generally result in more accurate efficiency estimates. In our dataset, 8 primers were removed due to an insignificant linear slope.
where is the estimated slope, t0.025;n-2is the 2.5% quantile of Student's t-distribution with n - 2 degrees of freedom, and SE( ) is the standard error of . This analysis used 3 or 4 dilution levels for each primer (n = 3 or 4), so we use a t-distribution with 1 or 2 degrees of freedom.
Only keeping those primers with a negative and significant slope ensures that the condition β upper < 0 is satisfied.
Identifying Unreliable Primers
Bartlett's test of heteroscadasticity (unequal variability within groups), applied to the residuals of each linear model fit by (1), is highly significant (p < 0.0001). This indicates that certain aberrations in efficiency estimation may be due to inherent primer reliability issues, rather than standard experimental and residual error. Purging those primers with missing values and an insignificant slope can be considered a first step to identify primer-pairs yielding unreliable data, but further analysis is warranted. Here we introduce transformed confidence intervals as a novel measure of primer reliability
The length of the transformed confidence interval E upper - E lower is used to quantify the precision of the estimated amplification efficiency . To identify those primers with unreliable data, we first calculate the interquartile range (IQR) of the transformed CI lengths as the difference betweeen the 75th and 25th percentiles (quantiles). Any primer with a transformed CI length higher than 1.5 × IQR above the 75th percentile is identified as unreliable. Such a procedure (the "1.5 × IQR" rule) is commonly used to identify outliers. In our dataset, 7 primers are identified using this procedure.
Most of those primers identified for large transformed CI's tend to have lower R2 values, whereas there is no correlation with primer pair efficiency. Hence, a low efficiency primer pair may indeed be very reliable and an apparently good efficiency primer may be the result of a bad fit. This demonstrates that considering a measure of variation for each efficiency estimate adds value over prior methods that consider efficiency as the sole criterion for primer performance.
Comparing Primer Efficiencies
Even after purging the data of unreliable primers, estimates of primer efficiency vary, from E = 1.7 to E > 2. It has been shown that rather than improving accuracy, individual corrections to efficiency can in fact exaggerate artificial differences in expression . Therefore, we consider the hypothesis that there is no underlying difference in primer efficiencies, and observed differences are due to residual error.
The F-statistic for the interaction between primer and log dilution log2(dil) is highly significant (p = 0.0017). This indicates that differences among slope coefficients, hence the resulting efficiency estimates, are not merely due to residual error.
This significant difference in primer efficiencies shows that simply using the average efficiency (E = 1.90) among all primers is inappropriate for comparing expression levels. An alternative approach is to use the individually determined efficiency value for each primer. Both methods generate errors on efficiency, which may be propogated through the calculation of expression quantities .
A trade-off exists between the lack of precision in not recognizing differences in primer efficiency, and the computational complexity and danger of over-fitting when using individually determined efficiencies for each primer. Here we introduce a novel, third alternative, which finds a useful balance between the two extreme approaches. We first cluster primer-pairs based on their estimated efficiency values, then use only a measure of the average efficiency within each cluster for subsequent adjustments. Rather than a single efficiency, i.e. no correction, we use 5-10 different efficiencies, based on clustering to obtain adjusted C T values. Rather than 96-1024 individual, but computational unreliable, efficiencies, we use 5-10 different cluster-derived efficiencies, knowing that within each cluster, there exists no significant difference between individual primer pairs.
Note that the majority of efficiencies, even across clusters, still are between E = 1.8 and E = 2.0. This is expected since we started with a "good" primerset based on sequence criteria . Still we are able to subdivide the "efficiency space" 1.8 to 2.0 into four clusters and thus improve overall performance. Furthermore, we are able to use even the relatively divergent primers pairs with E < 1.8 and E > 2 (due to repeats in the target sequence) for further analysis thus maximizing usability. The clustering method described is robust in the sense that significant differences in primer efficiencies should be reflected in the number of clusters. Both the magnitude of observed efficiency differences and precision of those efficiency estimates are accounted for. Arrays where the observed differences in primer efficiency are not significant are expected to result in few clusters, whereas if efficiencies differ greatly and can be measured precisely the method will tend to identify many clusters.
Comparison of Methods
For replications of the same experiment with different initial dilutions, we expect the AFC to be the same. However, an inaccurate assumed efficiency E will lead to different AFC values. For each primer, we calculate the coefficient of variation for the AFC values corresponding to the four dilution levels. The coefficient of variation (CoV) is defined as the standard deviation of the four values, divided by their mean.
Comparing Relative Expressions
This insight is captured for group-wise comparison in the REST  software package, which also incorporates error estimates and tests of significance, while allowing for manual input of PCR efficiencies.
Here, is the estimated number of cycles required under perfect efficiency, so that the true fold difference is given by . For example, if a reaction has perfect efficiency (E = 2) with observed C T = 24, while another has sub-perfect efficiency (E= 1.8) with observed C T = 28.5, we calculate = 24 for both reactions, indicating no difference in expression. A practical advantage of using is that the data format of the original set-up remains the same. The expression data remain log transformed and can be used directly in any microarray profiling software such as Eisen's original clustering program . Calculating is a preprocessing step that does not change the runtime or memory requirements of the subsequent analysis programs.
Adjusted Relative Expression.
Adjusted Relative Expression
Adjusted C T :
As array-based measurements for DNA, mRNA and microRNA levels migrate into the mainstay of molecular biology, failure to carefully consider the efficiency of each individual reaction or assay can lead to significant measurement inaccuracies. Yet, explicitly calculating and considering individual assay characteristics is not feasible even for low-density arrays. This is an important problem particularly for real-time QPCR based arrays, but it applies to any type of microarray. QPCR primers differ significantly both in reliability and amplification efficiency, hence they need to be experimentally validated. (i) We identified transformed confidence intervals as a useful means to assess and interpret the reliability of an efficiency estimate. Transformed confidence intervals provide a novel, independent measure in addition to calculation of primer efficacy E, with which to assess primer/probe quality. (ii) After purging unreliable estimates we propose a robust clustering algorithm to group efficiencies, reducing computational complexity and potential over-fitting. Our results suggest that use of a limited number of clustering-based efficiencies is comparable to use of individually determined efficiencies for each primer or probe.
This work was supported by funding from the University Cancer Research Fund, the AIDS malignancies clinical trials consortium (AMC), and the NIH (DE018304).
- Schmittgen T, Jiang J, Liu Q, Yang L: A high-throughput method to monitor the expression of microRNA precursors. Nucleic Acids Res 2004, 32: e43. 10.1093/nar/gnh040View ArticlePubMedPubMed Central
- Dittmer D: Transcription profile of Kaposi's sarcoma-associated herpesvirus in primary Kaposi's sarcoma lesions as determined by real-time PCR arrays. Cancer Res 2003, 63: 2010–5.PubMed
- O'Hara A, Vahrson W, Dittmer D: Gene alteration and precursor and mature microRNA transcription changes contribute to the miRNA signature of primary effusion lymphoma. Blood 2008, 111: 2347–53. 10.1182/blood-2007-08-104463View ArticlePubMedPubMed Central
- O'Hara A, Chugh P, Wang L, Netto E, Luz E, Harrington W, Dezube B, Damania B, Dittmer D: Pre-micro RNA signatures delineate stages of endothelial cell transformation in Kaposi sarcoma. PLoS Pathog 2009, 5: e1000389. 10.1371/journal.ppat.1000389View ArticlePubMedPubMed Central
- Gaur A, Jewell D, Liang Y, Ridzon D, Moore J, Chen C, Ambros V, Israel M: Characterization of microRNA expression levels and their biological correlates in human cancer cell lines. Cancer Res 2007, 67: 2456–68. 10.1158/0008-5472.CAN-06-2698View ArticlePubMed
- Yuan J, Reed A, Chen F, Stewart C: Statistical analysis of real-time PCR data. BMC Bioinformatics 2006, 7: 85. 10.1186/1471-2105-7-85View ArticlePubMedPubMed Central
- Karlen Y, McNair A, Persequers S, Mazza C, N M: Statistical significance of quantitative PCR. BMC Bioinformatics 2007, 8: 131. 10.1186/1471-2105-8-131View ArticlePubMedPubMed Central
- Bar T, Stahlberg A, Muszta A, M K: Kinetic Outlier Detection (KOD) in real-time PCR. Nucleic Acids Res 2003, 31: e105. 10.1093/nar/gng106View ArticlePubMedPubMed Central
- Pfaffl M, Graham W, Dempe L: Relative expression software tool (REST ©) for group-wise comparison and statistical analysis of relative expression results in real-time PCR. Nucleic Acids Res 2002, 30: e36. 10.1093/nar/30.9.e36View ArticlePubMedPubMed Central
- Pfaffl M: A new mathematical model for relative quantification in real-time RT-PCR. Nucleic Acids Res 2001, 29: e45. 10.1093/nar/29.9.e45View ArticlePubMedPubMed Central
- Papin J, Vahrson W, Hines-Boykin R, Dittmer D: Real-time quantitative PCR analysis of viral transcription. Methods Mol Biol 2005, 292: 449–80.PubMed
- Fakhari F, Dittmer D: Charting latency transcripts in Kaposi's sarcoma-associated herpesvirus by whole-genome real-time quantitative PCR. J Virol 2002, 76: 6213–23. 10.1128/JVI.76.12.6213-6223.2002View ArticlePubMedPubMed Central
- Hilscher C, Vahrson W, DP D: Faster quantitative real-time PCR protocols may lose sensitivity and show increased variability. Nucleic Acids Res 2005, 33: e182. 10.1093/nar/gni181View ArticlePubMedPubMed Central
- R Development Core Team: R Foundation for Statistical Computing, Vienna, Austria. R: A Language and Environment for Statistical Computing 2009. [ISBN 3–900051–07–0] [http://www.R-project.org] [ISBN 3-900051-07-0]
- Schefe J, Lehmann K: Quantitative real-time RT-PCR data analysis: current concepts and the novel "gene expression's Ct difference" formula. J Mol Med 2006, 84: 901–910. 10.1007/s00109-006-0097-6View ArticlePubMed
- Peirson S, Butler B, Foster R: Experimental validation of novel and conventional approaches to quantitative real-time PCR data analysis. Nucleic Acids Res 2003, 31: e73. 10.1093/nar/gng073View ArticlePubMedPubMed Central
- Hellemans J, Mortier G, Paepe A, Speleman F, J V: qBase relative quantification framework and software for management and automated of real-time quantitative PCR data. Genome Biology 2007, 8: R19. 10.1186/gb-2007-8-2-r19View ArticlePubMedPubMed Central
- Eisen M, Spellman P, Brown P, Botstein D: Cluster analysis and display of genome-wide expression patterns. Proc Natl Acad Sci 1998, 95: 14863–8. 10.1073/pnas.95.25.14863View ArticlePubMedPubMed Central
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.