BMC Bioinformatics BioMed Central Correspondence

Background Users of microarray technology typically strive to use universally acceptable data analysis strategies to determine significant expression changes in their experiments. One of the most frequently utilised methods for gene expression data analysis is SAM (significance analysis of microarrays). The impact of selection thresholds, on the output from SAM, may critically alter the conclusion of a study, yet this consideration has not been systematically evaluated in any publication. Results We have examined the effect of discrete data selection criteria (qualification criteria for inclusion) and response thresholds (out-put filtering) on the number of significant genes reported by SAM. The use of a reduced data set by applying arbitrary restrictions vis-à-vis abundance calls (e.g. from D-chip) or application of the fold change (FC) option within SAM (named the FC hurdle hereafter), can substantially alter the significant gene list when running SAM in Microsoft Excel. We determined that for a given final FC criteria (e.g. 1.5 fold change) the FC hurdle applied within Microsoft Excel SAM alters the number of reported genes above the final FC criteria. The reason is that the FC hurdle changes the composition of the control data set, such that a different significance level (q-value) is obtained for any given gene. This effect can be so large that it changes subsequent post hoc analysis interpretation, such as ontology overrepresentation analysis. Conclusion Our results argue for caution when using SAM. All data sets analysed with SAM could be reanalysed taking into account the potential impact of the use of arbitrary thresholds to trim data sets before significance testing.


Background
During a large scale analysis of data derived using the Affymetrix MOE430 murine DNA microarray [1], we detected striking differences in the resulting set of expressed genes depending on whether we were using one or another release of the microarray probe annotations as distributed by Affymetrix (NetAffx [2]).This is due to probes that point to different genes in different versions of the NetAffx data (see for example, the assignment for probe set 1433436_s_at in Table 1).Considering that gene names are broadly used by researchers when reporting microarray analysis results and in order to assess the magnitude of these changes, we measured their kind and extent through the history of all annotation files using as example the Affymetrix MOE430A/B chips.
Affymetrix DNA microarrays include probes for the detection of target sequences that are mainly based on UniGene clusters [3].UniGene is a database of gene-oriented clusters of GenBank sequences, where in addition to sequences of well-characterized genes, hundreds of thousands of novel expressed sequence tag sequences (ESTs) have been included.Affymetrix probe sets are annotated according to their related current records in UniGene and LocusLink, including genomic location, gene symbol, and function description, when available (NetAffx database, [2]).
We obtained all 8 NetAffx releases for the MOE430A/B microarray, dated from 2003 March 17th until 2004 June 6th (kindly provided by Marco Raposo, Affymetrix).First, we observed that there was at least one gene name change for 13,699 of the approximately 45,000 probe sets included in the MOE430A/B chips.Many of these changes were simply probe sets initially without a gene name that were eventually associated to one.This reflects a general improvement in the functional annotation of the human genome.Other changes could be explained by the use of synonymous gene symbols.However, according to a table of synonymous gene symbols that we extracted from the LocusLink gene database [4], there was still a total of 2277 probe sets with gene name changes that could not be explained by the use of a synonym.This represents about the 5% of the total of probe sets in the chip.
The underlying problem is exemplified in Table 1, where it can be seen that at least one probe must have been temporarily assigned to the wrong transcript.These inconsistencies can be detected when two probe sets attached to the same gene in one version of the annotations are attached to different gene names in another version.Table 2 indicates the number of inconsistencies by pairs of probe sets observed from one version of NetAffx annotations to the next, which amounts to thousands.This explains the variation in the biological interpretation of an Affymetrix microarray experiment depending on the version of the NetAffx annotations used.
The design of DNA microarray probe sets is often based on assembled groups of expressed sequences observed as ESTs or cDNAs, and might represent partial transcripts.Additional evidence in the form of new sequences, or even new gene predictions, can modify the preliminary assignment (for example, by discovering that two ESTs that were considered to be representing different mRNA transcripts are actually part of the same one).Therefore, information assigned to a probe based on gene predictions (such as a gene name) can be considered non-static and might change over time.Although, one can expect annotations will improve over time due to more accurate genomic assemblies, the changes will still occur for a while since a large fraction of genes are still predicted.Probe sequences constitute the only static information attached to the microarray: this information is inherent to the design of the microarray and will not change over time.This was pointed out in the manuscript that describes the NetAffx annotation files [2] but currently there is no visible warning or reminder in the Affymetrix website.
It happens that, although these are implicitly well known facts in the bioinformatics community, experimental users of microarrays are not so aware of the problem, probably because the surprisingly large extent of these changes has not been pointed out before.For example, the recent letter from the Microarray Gene Expression Data Society [5] explains that deposition of microarray data in public databases assures data persistence, integration, accessibility, and data standardization, but misses the problem of variable gene structure.There are recent publications that deal with the analysis of relations between Affymetrix probe sets and gene sequences [6][7][8], but they do no report the extent of the variation of these relations along time as we have done here.This latter fact, which An example of a pair of probe sets inconsistently annotated across the history of the eight NetAffx annotation files.Probe sets 1433436_s_at and 1419113_at were both assigned to gene Ap1g2, the gamma subunit of adaptor protein complex AP-1 in versions 4 and 5.This is a Golgi apparatus gene involved in protein transport.Thtpa is a hydrolase enzyme, the thiamine triphosphatase.In our experimental data, 1433436_s_at was detected as present and 1419113_at as absent.NetAffx releases of October 9 th and December 11 th 2003 would suggest that Ap1g2 was expressed, while any other release would give the opposite result.could convince many microarray users to send their data to public databases, has not been well publicized.
Deposition of microarray data in public databases is much more than just making the data public, but to making them really of use to the scientific community.Those databases include the descriptions of probe sequences and update constantly the non-static information associated to them, thus allowing the re-interpretation of the data and solving the problem we presented here.

Table 2 : Number of split and joined probe set pairs between consecutive versions of NetAffx.
Splits represent the number of pairs of probe sets that point to the same gene name in one NetAffx release but to a different gene name from each other in the following release.Joins represent the number of pairs of probe sets that point to different gene names from each other in a release but to the same gene name in the following release.For this computation, all probes with no gene name were considered as associated to a different gene name.Dates of the NetAffx versions are given in Table1.
available free of charge to the entire biomedical community peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours -you keep the copyright Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp BioMedcentral BMC Bioinformatics 2005, 6:183 http://www.biomedcentral.com/1471-2105/6/183