Skip to main content

Evaluating different methods of microarray data normalization

Abstract

Background

With the development of DNA hybridization microarray technologies, nowadays it is possible to simultaneously assess the expression levels of thousands to tens of thousands of genes. Quantitative comparison of microarrays uncovers distinct patterns of gene expression, which define different cellular phenotypes or cellular responses to drugs. Due to technical biases, normalization of the intensity levels is a pre-requisite to performing further statistical analyses. Therefore, choosing a suitable approach for normalization can be critical, deserving judicious consideration.

Results

Here, we considered three commonly used normalization approaches, namely: Loess, Splines and Wavelets, and two non-parametric regression methods, which have yet to be used for normalization, namely, the Kernel smoothing and Support Vector Regression. The results obtained were compared using artificial microarray data and benchmark studies. The results indicate that the Support Vector Regression is the most robust to outliers and that Kernel is the worst normalization technique, while no practical differences were observed between Loess, Splines and Wavelets.

Conclusion

In face of our results, the Support Vector Regression is favored for microarray normalization due to its superiority when compared to the other methods for its robustness in estimating the normalization curve.

Background

DNA microarray technology is a powerful approach for genomic research, playing an increasingly important role in biomedical research. This technology yields simultaneous measurement of gene expression levels of thousands of genes, allowing the analysis of differential gene expression patterns under different conditions such as disease (pathological) states or treatment with different chemotherapeutic drugs. Due to small differences in RNA quantities and fluctuations generated by the technique, the intensity levels may vary from one replicate to the other due to effects which are unrelated to the genes, requiring data normalization before they can be compared.

Therefore, normalization is an important step for microarray data analysis. The purpose of data normalization is to minimize the effects caused by technical variations and, as a result, allow the data to be comparable in order to find actual biological changes. Several normalization approaches have been proposed, most of which derive from studies using two-color spotted microarrays. Some authors proposed normalization of the hybridization intensity ratios; others use global, linear methods, while others use local, non-linear methods. Several authors suggested using the spike-in controls, housekeeping genes, or invariant genes [17].

Recently, some authors suggested the use of non-linear normalization methods [810] which are believed to be superior to the above mentioned approaches. The locally weighed regression Lowess procedure [11] has been widely used for this purpose and implemented by several microarray analysis software packages [12, 13], but similar methods are suggested such as Splines [14, 15] and Wavelets [16].

Here, we compare three different well-known microarray data normalization methods, namely: Loess Regression (LR), Splines Smoothing (SS) and Wavelets Smoothing (WS). In addition, we propose two different normalization approaches, called Kernel Regression (KR) [17, 18] and Support Vector Regression (SVR) [19], which, to the best of our knowledge, have yet to be applied for microarray normalization. In order to assess the most appropriate normalization technique, benchmark studies were carried out using data derived from CodeLink™ mouse microarray experiments [20], generated at our Cell and Molecular Biology Laboratory (Chemistry Institute, University of São Paulo).

Results

We sought to highlight the performance of five different methods of microarray normalization, namely: Loess, Splines, Wavelets, Kernel and Support Vector Regression in a simulated microarray and in an actual CodeLink™ microarray platform, which comprised ten thousand mouse genes. Although we have focused on the use of simulated two-color cDNA microarray data analysis, our discussions are also applicable to the single-color oligonucleotide microarrays.

The artificial microarrays composed by ten thousand spots were generated using the model proposed by Balagurunathan et al. (2002) [21]. The parameters used were: ( a 0 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGymaedaaaaa@3002@ = 0, a 1 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGymaedaaaaa@3004@ = 1001/0.7, a 2 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGymaedaaaaa@3006@ = -0.7, a 3 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGymaedaaaaa@3008@ = 1) and ( a 0 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGOmaidaaaaa@3004@ = 0, a 1 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGOmaidaaaaa@3006@ = 1001/0.9, a 2 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGOmaidaaaaa@3008@ = -0.9, a 3 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGOmaidaaaaa@300A@ = 1) for sinusoid shape, ( a 0 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGymaedaaaaa@3002@ = 0, a 1 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGymaedaaaaa@3004@ = 500 a 2 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGymaedaaaaa@3006@ = -1, a 3 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGymaedaaaaa@3008@ = 1) and ( a 0 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGOmaidaaaaa@3004@ = 0, a 1 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGOmaidaaaaa@3006@ = 10, a 2 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGOmaidaaaaa@3008@ = -1, a 3 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGOmaidaaaaa@300A@ = 1) for banana shape and, ( a 0 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGymaedaaaaa@3002@ = 0, a 1 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGymaedaaaaa@3004@ = 10, a 2 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGymaedaaaaa@3006@ = -1, a 3 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGymaedaaaaa@3008@ = 1) and ( a 0 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGimaadabaGaeGOmaidaaaaa@3004@ = 0, a 1 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGymaedabaGaeGOmaidaaaaa@3006@ = 1001/0.7, a 2 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeGOmaidabaGaeGOmaidaaaaa@3008@ = -0.7, a 3 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqydaqhaaWcbaGaeG4mamdabaGaeGOmaidaaaaa@300A@ = 1) for mixed shape. Gene expression was generated by an exponential distribution with parameter λ = 1/3000 and the outliers were generated by a Beta distribution with parameters B(1.7,4.8). For more details, see Balagurunathan et al. (2002).

The smoothing parameters used in each dataset are described in Table 1. For SVR, we tested a range of values and, as a result, we selected ε = 0.01 and C = 4 as the most adequate one. It is important to highlight that the parameters are arbitrary; therefore, we chose the optimum parameters for each method, i.e., the one which resulted in the lowest mean square error. In Figure 1 are described the mean square errors for each normalization method applied to three different simulated microarrays with no outliers.

Table 1 Smoothing parameters used for each microarray dataset. For Loess, it is the span value, for Splines and Wavelets it is the number of functions, for Kernel and SVR it is the maximum value minus the minimum value multiplied by the number described in the table.
Figure 1
figure1

The minimum mean square error for three different simulated microarray datasets. From left to right: 1) sinusoid shape; 2) banana shape; 3) mix shape. The Kernel regression was not included in this figure because its MSE is 103 orders of magnitude greater than the other normalization methods.

In order to compare the perturbation caused by the presence of outliers and the robustness of each normalization method, we randomly inserted 5, 10, 15, 20 and 40% of outliers (genes which display very high differential expression) at three different expression levels (low, medium, high), and the respective mean square errors between the regression curve and the actual curve (the function from which the microarray was generated) was calculated. This step was repeated 100 times to estimate the average sum of the squared errors and their variance. The Wilcoxon and the Kolmogorov-Smirnov tests were performed in order to determine whether the five regression methods differ from one another in any significant manner.

A high performance normalization technique should yield unbiased corrections and corrections with the smallest standard deviation.

Comparison of the results presented in Table 2, 3 and 4 shows no important difference between LR, SS and WS. Although the non-parametric KR method has been successfully applied in econometrics data analysis [22], it displayed a poor performance for microarray normalization, probably because it is highly sensitive to outliers [23].

Table 2 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the low expression levels condition.
Table 3 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the medium expression levels condition.
Table 4 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the high expression levels conditions.

Upon analyzing Table 2, it is possible to observe, in the case of sinusoid shape, that when outliers are inserted in regions of low gene expression, SVR, WS, SS, LR and KR, in this order, have the lowest to the highest mean square error, being statistically different (p value < 0.001) from one another. For the banana and mixed shapes, LR and SS presented a lower MSE than WS. In Table 3, it is interesting to note that when outliers are inserted in regions of medium gene expression, i.e., high density of genes, the order of performance remains the same as in Table 2 and SVR displays a mean square error which is significantly different from the others (p value < 0.001). LR and SS showed no significant difference (p value > 0.05) and KR is significantly worse than the other methods (p value < 0.001). In Table 4, the outliers are inserted in a high gene expression region. Once more, the trend is maintained, namely, KR is the most affected by outliers (p value < 0.001) and no differences between SS and WS (p value > 0.05) were observed for the sinusoid shape. For the other two shapes, LR and SS were better than WS (p value < 0.001).

In all three cases (outliers at low, medium and high gene expression), SVR is the affected by outliers (p value < 0.001), independently the microarray's shape. In addition, SVR yields the smallest standard deviation, followed by LR, SS, WS, with KR displaying the largest deviation. In addition, the five methods were applied to actual microarray data, with outliers inserted artificially, and the results were the same when compared to those obtained from artificial microarray experiments.

In Figure 2, we illustrate the performance of the five normalization methods applied to actual microarray data, without the insertion of artificial outliers. A small difference could be observed in the normalization curves in which the genes displayed low and high expression, due to the low quantity of genes and the high variance.

Figure 2
figure2

Fitted normalization curves for actual cDNA microarray data using the five different normalization methods (Loess, Splines, Wavelets, Kernel, SVR).

Discussion

By analyzing the extent to which the outliers could disturb the regression curve, we observed that KR is more highly sensitive to outliers than LR, SS and WS in all three cases (outliers in low, medium and high expression). In all three cases, SVR is shown to be the least affected.

The superior performance of Splines, when compared to KR, may be explained by the degree of smoothing, which varies according to the density of points, differently from KR, which has a fixed window size. Wavelet also has a slightly better performance than KR, probably due to multi-resolution properties. In general, SS and WS presented similar performance when we compared the median of the mean square error using the Wilcoxon test. However, when we used the Kolmogorov-Smirnov test, they presented a statistically significant difference (p value < 0.001). SS and WS constitute somewhat better normalization techniques than LR when we analyzed the sinusoid shape, but, for the other two shapes, LR is better than SS and WS. For practical purposes, the differences between them in terms of disturbance by outliers are too small to be of any concern.

The SVR method is shown to be very robust to outliers presented at different gene expression levels, becoming the best normalization technique to identify actual differentially expressed genes.

One well-known problem in identifying differentially expressed genes is normalizing genes displaying low expression levels, due to the low quantity of the corresponding transcripts and the high spot intensity variance. An equivalent problem occurs with genes presenting very high expression levels due to the low frequency of these genes. Once more, under these conditions, the SVR method is shown to be better than other currently used methods.

We performed the same tests for five other pairs of CodeLink™ microarrays and the results obtained were the same: the SVR is the most robust to outliers and the KR method is the worst method, being highly sensitive to differentially expressed genes and yielding poor regression curves.

Other methods, which are also robust to outliers and are based on a new regression method called two-way semi-linear model [2427] have also been applied for microarray data normalization. This new approach developed in the last few years, deserves further studies, which we are planning to undertake in the future.

Conclusion

We have proposed a new approach to normalize microarray data and tested this SVR method by benchmark studies and by several simulations. The results obtained with SVR were superior than those obtained with some widely used normalization techniques such as LR, SS and WS. SVR is shown to be more robust to outliers even at very low and very high gene expression levels, being useful to identify differentially expressed genes. Even tested in different microarray shapes, SVR was superior to the other methods, while LR, SS and WS presented similar performances. Therefore, we have demonstrated that SVR is feasible and very promising for microarray data normalization.

Methods

Simulation

The program which generates the artificial microarray and the analyses were implemented in R, a language for statistical computing [28]. This script may be downloaded at: [29].

CodeLink™ microarray

Cell lysis and RNA extraction

Cell cultures were lysed with guanidine isocyanate and RNA was purified by of the cell lysates on a cesium chloride cushion (Chirgwin el al, 1979). Absorbance ratio at 260/280 nm was used to assess the RNA purity, a ratio of 1.8 – 2.0 indicating adequate purity.

Labeling and purification of targets

RNA samples were prepared and processed according to protocols supplied by the manufacturer (Amersham Biosciences). Briefly, cDNAs were synthesized from purified RNA (2 μg) and control bacterial mRNAs. Samples were purified using the QIAquick Spin kit (Qiagen) and concentrated by SpeedVac. Concentrated pellets were used in a biotinylated-UTP based cRNA synthesis using the CodeLink™ Expression Assay Reagent Kit (Amersham). Labeled cRNAs were purified using the RNeasy kit (Qiagen) and fragmented with supplied solution at 94°C for 20 min.

Hybridization and washing of arrays

Fragmented biotin-labeled cRNAs (10 μg) were incubated with CodeLink™ bioarrays and shaken (300 rpm) for 20 h. The bioarrays were then washed and incubated with Cy5-Streptavidin (30 min). Scanning of the bioarrays was performed in a GenePix 4000 B Array Scanner (Axon Instruments) and the data were collected using the CodeLink™ System Software (Amersham), which provided the raw data and invalidated data from irregular spots.

Loess regression

Consider we have n measurements, for each of which the response expected is y i and let x i be the predictor, where x is the log intensity of one microarray and y is the log intensity of the other one, in case we are analyzing a single-color microarrays. Whether the microarray is a two-color platform, x is the log of one dye intensity and y is the log of the other dye intensity.

In this model, they are supposed to be related by

y i = g(x i ) + ε i     (1)

where g is the regression function and ε i is a random error. The idea of local regression is that near x = x0, the regression function g(x) can be locally approximated by the value of a function in some specified parametric class. Such a local approximation is obtained by fitting a regression surface to the data points within a chosen neighborhood of the point x0.

In this method, weighed least squares are used to fit linear or quadratic functions of the predictors at the centers of the neighborhoods. The radius of each neighborhood is chosen so that the neighborhood contains a specified percentage of the data points. The fraction of the data, called the smoothing parameter, in each local neighborhood is weighted by a smooth decreasing function of their distance from the center of the neighborhood [30].

B-Splines smoothing

Due to its simple structure and good approximation properties, polynomials are widely used in practice for approximating functions [31, 32]. Let x and y as defined above and

( x y ) + 0 = { 1 , x y 0 , x < y ( 2 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGGOaakcqWG4baEcqGHsislcqWG5bqEcqGGPaqkdaqhaaWcbaGaey4kaScabaGaeGimaadaaOGaeyypa0ZaaiqabeaafaqabeGabaaabaGaeGymaeJaeiilaWIaemiEaGNaeyyzImRaemyEaKhabaGaeGimaaJaeiilaWIaemiEaGNaeyipaWJaemyEaKhaaaGaay5EaaGaaCzcaiaaxMaadaqadaqaaiabikdaYaGaayjkaiaawMcaaaaa@4683@

and

( x y ) + m 1 = { ( x y ) m 1 , x y , m > 1 0 , x < y ( 3 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGGOaakcqWG4baEcqGHsislcqWG5bqEcqGGPaqkdaqhaaWcbaGaey4kaScabaGaemyBa0MaeyOeI0IaeGymaedaaOGaeyypa0ZaaiqabeaafaqaaeGabaaabaGaeiikaGIaemiEaGNaeyOeI0IaemyEaKNaeiykaKYaaWbaaSqabeaacqWGTbqBcqGHsislcqaIXaqmaaGccqGGSaalcqWG4baEcqGHLjYScqWG5bqEcqGGSaalcqWGTbqBcqGH+aGpcqaIXaqmaeaacqaIWaamcqGGSaalcqWG4baEcqGH8aapcqWG5bqEaaaacaGL7baacaWLjaGaaCzcaiabcIcaOiabiodaZiabcMcaPaaa@5554@

Therefore, let

... ≤ y-1y0y1y2 ≤ ...     (4)

be a sequence of real numbers. Given integers i and m > 0, we define

Q i m ( x ) = { ( 1 ) m [ y i , ... , y i + m ] ( x y ) + m 1 , i f y i < y i + m 0 , o t h e r w i s e ( 5 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGrbqudaqhaaWcbaGaemyAaKgabaGaemyBa0gaaOGaeiikaGIaemiEaGNaeiykaKIaeyypa0ZaaiqabeaafaqaaeGabaaabaGaeiikaGIaeyOeI0IaeGymaeJaeiykaKYaaWbaaSqabeaacqWGTbqBaaGccqGGBbWwcqWG5bqEdaWgaaWcbaGaemyAaKgabeaakiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiabdMha5naaBaaaleaacqWGPbqAcqGHRaWkcqWGTbqBaeqaaOGaeiyxa0LaeiikaGIaemiEaGNaeyOeI0IaemyEaKNaeiykaKYaa0baaSqaaiabgUcaRaqaaiabd2gaTjabgkHiTiabigdaXaaakiabcYcaSiabdMgaPjabdAgaMjaaykW7caaMc8UaemyEaK3aaSbaaSqaaiabdMgaPbqabaGccqGH8aapcqWG5bqEdaWgaaWcbaGaemyAaKMaey4kaSIaemyBa0gabeaaaOqaaiabicdaWiabcYcaSiabd+gaVjabdsha0jabdIgaOjabdwgaLjabdkhaYjabdEha3jabdMgaPjabdohaZjabdwgaLbaaaiaawUhaaiaaxMaacaWLjaGaeiikaGIaeGynauJaeiykaKcaaa@76CA@

for all real x. We call Q i m MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGrbqudaqhaaWcbaGaemyAaKgabaGaemyBa0gaaaaa@30C2@ the m th order B-Spline associated with the knots y i ,..., yi + m.

For m = 1, the B-Spline associated with y i <yi + 1is particularly simple. It is the piecewise constant function

Q i 1 ( x ) = { 1 y i + 1 y i , y i x y i + 1 0 , o t h e r w i s e . ( 6 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGrbqudaqhaaWcbaGaemyAaKgabaGaeGymaedaaOGaeiikaGIaemiEaGNaeiykaKIaeyypa0ZaaiqabeaafaqaaeGabaaabaWaaSaaaeaacqaIXaqmaeaacqWG5bqEdaWgaaWcbaGaemyAaKMaey4kaSIaeGymaedabeaakiabgkHiTiabdMha5naaBaaaleaacqWGPbqAaeqaaaaakiabcYcaSiabdMha5naaBaaaleaacqWGPbqAaeqaaOGaeyizImQaemiEaGNaeyizImQaemyEaK3aaSbaaSqaaiabdMgaPjabgUcaRiabigdaXaqabaaakeaacqaIWaamcqGGSaalcqWGVbWBcqWG0baDcqWGObaAcqWGLbqzcqWGYbGCcqWG3bWDcqWGPbqAcqWGZbWCcqWGLbqzcqGGUaGlaaaacaGL7baacaWLjaGaaCzcaiabcIcaOiabiAda2iabcMcaPaaa@605C@

In our analysis, we applied the cubic Splines, i.e., Splines of order 3.

We can also give explicit formulate for Q i m MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGrbqudaqhaaWcbaGaemyAaKgabaGaemyBa0gaaaaa@30C2@ in case either y i or yi + mis a knot of multiplicity m.

Wavelet smoothing

The Wavelet transform is a relatively new approach and has some similarities with the Fourier transform. Wavelets differ from Fourier methods in that they allow the localization of a signal in both time and frequency. In the wavelet theory, a function is represented by an infinite series expansion in terms of dilated and translated version of a basic function ψ called the "mother' Wavelet. A Wavelet transformation leads to an additive decomposition of a signal into a series of different components describing smooth and rough features of the signal.

The term Wavelets means small curves, therefore, they are oscillations that rapidly decay. As the B-Splines functions system, the Wavelets functions ψ(t) can be used to generate a function basis for certain spaces [33]. An ortonormal basis can be generated by dyadic dilations and translations of a mother Wavelet ψ(t), by

ψ j, k (t) = 2j/2ψ (2jj - k),     j, k Z (7)

Wavelets are functions which satisfy the following properties:

  1. i)
    ψ ( t ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaaGGaciab=H8a5bWcbaGaeyOeI0IaeyOhIukabaGaeyOhIukaniabgUIiYdGccqGGOaakcqWG0baDcqGGPaqkaaa@37E6@
    (1)

    dt = 0     (8).

ii) ψ ( t ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaaGGaciab=H8a5naabmaabaGaemiDaqhacaGLOaGaayzkaaaaleaacqGHsislcqGHEisPaeaacqGHEisPa0Gaey4kIipaaaa@37B3@ dt < ∞     (9).

iii) | Ψ ( ω ) | 2 d ω | ω | = 0 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaamaalaaabaWaaqWaaeaacqqHOoqwcqGGOaakcqaHjpWDcqGGPaqkaiaawEa7caGLiWoadaahaaWcbeqaaiabikdaYaaakiabdsgaKjabeM8a3bqaamaaemaabaGaeqyYdChacaGLhWUaayjcSdaaaaWcbaGaeyOeI0IaeyOhIukabaGaeyOhIukaniabgUIiYdGccqGH9aqpcqaIWaamaaa@4658@ , where the function Ψ(ω) is the Fourier transform of ψ(t)     (10).

iv) t j ψ ( t ) d t = 0 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaaiabdsha0naaCaaaleqabaGaemOAaOgaaGGacOGae8hYdKNaeiikaGIaemiDaqNaeiykaKIaemizaqMaemiDaqNaeyypa0JaeGimaadaleaacqGHsislcqGHEisPaeaacqGHEisPa0Gaey4kIipaaaa@3F97@ ,     j = 0,1,..., r - 1     for r ≥ 1 and | t r ψ ( t ) | d t < MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaamaaemaabaGaemiDaq3aaWbaaSqabeaacqWGYbGCaaacciGccqWFipqEcqGGOaakcqWG0baDcqGGPaqkaiaawEa7caGLiWoaaSqaaiabgkHiTiabg6HiLcqaaiabg6HiLcqdcqGHRiI8aOGaemizaqMaemiDaqNaeyipaWJaeyOhIukaaa@4354@     (11).

An important result is that any function f (t) with f 2 ( t ) < MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWdXbqaaiabdAgaMnaaCaaaleqabaGaeGOmaidaaOGaeiikaGIaemiDaqNaeiykaKIaeyipaWJaeyOhIukaleaacqGHsislcqGHEisPaeaacqGHEisPa0Gaey4kIipaaaa@3AFA@ can be expanded as

f ( t ) = j = k = c j , k ψ j , k ( t ) ( 12 ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGMbGzcqGGOaakcqWG0baDcqGGPaqkcqGH9aqpdaaeWbqaamaaqahabaGaem4yam2aaSbaaSqaaiabdQgaQjabcYcaSiabdUgaRbqabaaabaGaem4AaSMaeyypa0JaeyOeI0IaeyOhIukabaGaeyOhIukaniabggHiLdacciGccqWFipqEdaWgaaWcbaGaemOAaOMaeiilaWIaem4AaSgabeaaaeaacqWGQbGAcqGH9aqpcqGHsislcqGHEisPaeaacqGHEisPa0GaeyyeIuoakiabcIcaOiabdsha0jabcMcaPiaaxMaacaWLjaGaeiikaGIaeGymaeJaeGOmaiJaeiykaKIaeiOla4caaa@56A5@

In other words, any function f (t) can be represented by a linear combination of functions ψ j, k (t). The smoothing procedure can be carried out by an approximation, choosing a maximum resolution J (t) for j =1,2,..., J (t) and k = 1,2,..., 2j-1. Here, we considered the Mexican hat Wavelet [34] defined by

ψ ( t ) = ( 1 t 2 ) exp ( t 2 2 ) ( 13 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFipqEcqGGOaakcqWG0baDcqGGPaqkcqGH9aqpcqGGOaakcqaIXaqmcqGHsislcqWG0baDdaahaaWcbeqaaiabikdaYaaakiabcMcaPiGbcwgaLjabcIha4jabcchaWnaabmaabaWaaSaaaeaacqGHsislcqWG0baDdaahaaWcbeqaaiabikdaYaaaaOqaaiabikdaYaaaaiaawIcacaGLPaaacaWLjaGaaCzcaiabcIcaOiabigdaXiabiodaZiabcMcaPaaa@47F3@

rather than other functions such as Morlet or Shannon since they do not have an analytic formula.

The C jk coefficients are estimated via an ordinary least square regression. An important feature in the wavelets representation is that it allows the description of functions belonging to both Sobolev and Besov spaces [35].

Kernel regression

KR is one class of modeling methods that belongs to the smoothing methods family. It is part of the non-parametric regression methods. KR allows basing the prediction of a value on passed observations, and weighing the impact of past observations depending on how similar they are, compared to the current values of the explanatory variables.

The KR is one of the most widely used procedures in non-parametric curve estimation. Nadaraya (1964) and Watson (1964) proposed an estimator for the curve g given by

g h ( x ) = i = 1 n K h ( x X i ) Y i j = 1 n K h ( x X j ) ( 14 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGNbWzdaWgaaWcbaGaemiAaGgabeaakiabcIcaOiabdIha4jabcMcaPiabg2da9maalaaabaWaaabCaeaacqWGlbWsdaWgaaWcbaGaemiAaGgabeaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGUbGBa0GaeyyeIuoakiabcIcaOiabdIha4jabgkHiTiabdIfaynaaBaaaleaacqWGPbqAaeqaaOGaeiykaKIaemywaK1aaSbaaSqaaiabdMgaPbqabaaakeaadaaeWbqaaiabdUealnaaBaaaleaacqWGObaAaeqaaaqaaiabdQgaQjabg2da9iabigdaXaqaaiabd6gaUbqdcqGHris5aOGaeiikaGIaemiEaGNaeyOeI0IaemiwaG1aaSbaaSqaaiabdQgaQbqabaGccqGGPaqkaaGaaCzcaiaaxMaacqGGOaakcqaIXaqmcqaI0aancqGGPaqkaaa@5C79@

In our datasets, we used the Gaussian Kernel because it is symmetric and centralized in the mean.

In addition to being easy to compute, the Nadaraya-Watson estimator g h (x) is consistent. When h → 0 the estimated curve presents a large variability and when nh → ∞, we obtain an overly smooth curve [36]. The bandwidth h controls the smoothness degree of the estimated curve. It is easy to observe that this KR estimator is just a weighted sum of the observed responses Y i . The denominator ensures that the weights sum up to 1.

Support Vector Regression

SVR generalized algorithm is a non-linear regression from the Generalized Portrait algorithm developed in Russia by Vapnik and Lerner (1963) [37] and Vapnik and Chervonenkis (1964) [38]. It is based upon the statistical learning theory which has been developed by Vapnik and Chervonenkis (1974) [39]. In Bioinformatics, and, more specifically, in microarray data analysis, to the best of our knowledge, this algorithm has previously been used only once, by Hisanori et al. (2004), to extract relations between promoter sequences and strengths [40]. Here, we propose the use of SVR to normalize microarray data.

Let {(x1, y1),..., (x1, y1)} R × R be the gene expression data derived from microarray experiments, where x is the log intensity of one microarray and y is the log intensity of the other one, in case we are analyzing a single-color microarrays. When the microarray is a two-color platform, x is the log of one dye intensity and y is the log of the other dye intensity. In ε-SVR [41], the goal is to obtain a function f (x) that has at the most ε deviation from the y i for all the data, and is as flat as possible.

In the case of linear functions f :

f (x) = (wtx) + b with w Rn, b R     (15)

Flatness in (15) means

Minimize 1 2 w 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWcaaqaaiabigdaXaqaaiabikdaYaaadaqbdaqaaiabdEha3bGaayzcSlaawQa7amaaCaaaleqabaGaeGOmaidaaaaa@345B@

Constrained to  { y i ( w t x i ) b ε ( w t x i ) + b y i ε ( 16 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqqGdbWqcqqGVbWBcqqGUbGBcqqGZbWCcqqG0baDcqqGYbGCcqqGHbqycqqGPbqAcqqGUbGBcqqGLbqzcqqGKbazcqqGGaaicqqG0baDcqqGVbWBcqqGGaaidaGabeqaauaabaqaceaaaeaacqWG5bqEdaWgaaWcbaGaemyAaKgabeaakiabgkHiTmaabmaabaGaem4DaC3aaWbaaSqabeaacqWG0baDaaGccqWG4baEdaWgaaWcbaGaemyAaKgabeaaaOGaayjkaiaawMcaaiabgkHiTiabdkgaIjabgsMiJIGaciab=v7aLbqaamaabmaabaGaem4DaC3aaWbaaSqabeaacqWG0baDaaGccqWG4baEdaWgaaWcbaGaemyAaKgabeaaaOGaayjkaiaawMcaaiabgUcaRiabdkgaIjabgkHiTiabdMha5naaBaaaleaacqWGPbqAaeqaaOGaeyizImQae8xTdugaaiaaxMaacaWLjaGaeiikaGIaeGymaeJaeGOnayJaeiykaKcacaGL7baaaaa@684F@

In (16) there is a function f which, with ε precision, approximates all pairs (xi, y i ). But there are cases where it is necessary to allow for some errors. To solve this problem, one can introduce slack variables ξ i , ξ i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaH+oaEdaqhaaWcbaGaemyAaKgabaGaey4fIOcaaaaa@30E6@ to deal with unfeasible constraints of the optimization problem (16) arriving at the formulation stated in [41]

Minimize 1 2 w 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaWcaaqaaiabigdaXaqaaiabikdaYaaadaqbdaqaaiabdEha3bGaayzcSlaawQa7amaaCaaaleqabaGaeGOmaidaaaaa@345B@ + C ∑(ξ i + ξ i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaH+oaEdaqhaaWcbaGaemyAaKgabaGaey4fIOcaaaaa@30E6@ )

Constrained to  { y i ( w t x i ) b ε + ξ i ( w t x i ) + b y i ε + ξ i * ξ i , ξ i * 0 ( 17 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqqGdbWqcqqGVbWBcqqGUbGBcqqGZbWCcqqG0baDcqqGYbGCcqqGHbqycqqGPbqAcqqGUbGBcqqGLbqzcqqGKbazcqqGGaaicqqG0baDcqqGVbWBcqqGGaaidaGabeqaauaabaqadeaaaeaacqWG5bqEdaWgaaWcbaGaemyAaKgabeaakiabgkHiTmaabmaabaGaem4DaC3aaWbaaSqabeaacqWG0baDaaGccqWG4baEdaWgaaWcbaGaemyAaKgabeaaaOGaayjkaiaawMcaaiabgkHiTiabdkgaIjabgsMiJIGaciab=v7aLjabgUcaRiab=57a4naaBaaaleaacqWGPbqAaeqaaaGcbaWaaeWaaeaacqWG3bWDdaahaaWcbeqaaiabdsha0baakiabdIha4naaBaaaleaacqWGPbqAaeqaaaGccaGLOaGaayzkaaGaey4kaSIaemOyaiMaeyOeI0IaemyEaK3aaSbaaSqaaiabdMgaPbqabaGccqGHKjYOcqWF1oqzcqGHRaWkcqWF+oaEdaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGcbaGae8NVdG3aaSbaaSqaaiabdMgaPbqabaGccqGGSaalcqWF+oaEdaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaOGaeyyzImRaeGimaadaaaGaay5EaaGaaCzcaiaaxMaacqGGOaakcqaIXaqmcqaI3aWncqGGPaqkaaa@7CA1@

where the constant C > 0 is the trade-off between the amount up to which deviations larger than ε are tolerated, maintaining the flatness of f. This corresponds to dealing with the ε-insensitive loss function |ξ| ε :

| ξ | ε : = { i f | ξ | ε | ξ | ε o t h e r w i s e ( 18 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaabdaqaaGGaciab=57a4bGaay5bSlaawIa7amaaBaaaleaacqWF1oqzaeqaaOGaeiOoaOJaeyypa0ZaaiqabeaafaqaaeGabaaabaGaeGimaaJaeeiiaaIaemyAaKMaemOzay2aaqWaaeaacqWF+oaEaiaawEa7caGLiWoacqGHKjYOcqWF1oqzaeaadaabdaqaaiab=57a4bGaay5bSlaawIa7aiabgkHiTiab=v7aLjabbccaGiabd+gaVjabdsha0jabdIgaOjabdwgaLjabdkhaYjabdEha3jabdMgaPjabdohaZjabdwgaLbaaaiaawUhaaiaaxMaacaWLjaGaeiikaGIaeGymaeJaeGioaGJaeiykaKcaaa@5CD4@

It is necessary to construct a Lagrange function from the primal objective function and the corresponding constraints by introducing a dual set of variables. According to Mangasarian (1969) [42], McCormick (1983) [43], and Vanderbei (1997) [44] it follows that:

L : = 1 2 w 2 + C i = 1 l ( ξ i + ξ i * ) i = 1 l ( η i ξ i + η i * ξ i * ) i = 1 l α i ( ε + ξ i y i + ( w t x i ) + b ) α i * ( ε + ξ i * + y i ( w t x i ) b ) ( 19 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGmbatcqGG6aGocqGH9aqpdaWcaaqaaiabigdaXaqaaiabikdaYaaadaqbdaqaaiabdEha3bGaayzcSlaawQa7amaaCaaaleqabaGaeGOmaidaaOGaey4kaSIaem4qam0aaabCaeaadaqadaqaaGGaciab=57a4naaBaaaleaacqWGPbqAaeqaaOGaey4kaSIae8NVdG3aa0baaSqaaiabdMgaPbqaaiabcQcaQaaaaOGaayjkaiaawMcaaiabgkHiTaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdGcdaaeWbqaamaabmaabaGae83TdG2aaSbaaSqaaiabdMgaPbqabaGccqWF+oaEdaWgaaWcbaGaemyAaKgabeaakiabgUcaRiab=D7aOnaaDaaaleaacqWGPbqAaeaacqGGQaGkaaGccqWF+oaEdaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGSbaBa0GaeyyeIuoakiabgkHiTmaaqahabaGae8xSde2aaSbaaSqaaiabdMgaPbqabaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdGcdaqadaqaaiab=v7aLjabgUcaRiab=57a4naaBaaaleaacqWGPbqAaeqaaOGaeyOeI0IaemyEaK3aaSbaaSqaaiabdMgaPbqabaGccqGHRaWkdaqadaqaaiabdEha3naaCaaaleqabaGaemiDaqhaaOGaemiEaG3aaSbaaSqaaiabdMgaPbqabaaakiaawIcacaGLPaaacqGHRaWkcqWGIbGyaiaawIcacaGLPaaacqGHsisldaaeabqaaiab=f7aHnaaDaaaleaacqWGPbqAaeaacqGGQaGkaaaabeqab0GaeyyeIuoakmaabmaabaGae8xTduMaey4kaSIae8NVdG3aa0baaSqaaiabdMgaPbqaaiabcQcaQaaakiabgUcaRiabdMha5naaBaaaleaacqWGPbqAaeqaaOGaeyOeI0YaaeWaaeaacqWG3bWDdaahaaWcbeqaaiabdsha0baakiabdIha4naaBaaaleaacqWGPbqAaeqaaaGccaGLOaGaayzkaaGaeyOeI0IaemOyaigacaGLOaGaayzkaaGaaCzcaiaaxMaacqGGOaakcqaIXaqmcqaI5aqocqGGPaqkaaa@A784@

where L is the Lagrangian and η i , η i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWF3oaAdaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30C3@ , α i , α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ are Lagrange multipliers. Hence the dual variables in (19) have to satisfy

α i ( * ) , η i ( * ) 0 ( 20 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiikaGIaeiOkaOIaeiykaKcaaOGaeiilaWIae83TdG2aa0baaSqaaiabdMgaPbqaaiabcIcaOiabcQcaQiabcMcaPaaakiabgwMiZkabicdaWiaaxMaacaWLjaGaeiikaGIaeGOmaiJaeGimaaJaeiykaKcaaa@40A3@

Note that we refer to α i and α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ as α i ( ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiikaGIaey4fIOIaeiykaKcaaaaa@327B@ .

From the saddle point condition, the partial derivatives of L related to (w, b, ξ i , ξ i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaH+oaEdaqhaaWcbaGaemyAaKgabaGaey4fIOcaaaaa@30E6@ ) have to vanish for optimality.

b L = i = 1 l ( α i * α i ) = 0 ( 21 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGHciITdaWgaaWcbaGaemOyaigabeaakiabdYeamjabg2da9maaqahabaWaaeWaaeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaOGaeyOeI0Iae8xSde2aaSbaaSqaaiabdMgaPbqabaaakiaawIcacaGLPaaacqGH9aqpcqaIWaamcaWLjaGaaCzcaiabcIcaOiabikdaYiabigdaXiabcMcaPaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdaaaa@4931@
w L = w i = 1 l ( α i α i * ) x i = 0 ( 22 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGHciITdaWgaaWcbaGaem4DaChabeaakiabdYeamjabg2da9iabdEha3jabgkHiTmaaqahabaWaaeWaaeaaiiGacqWFXoqydaWgaaWcbaGaemyAaKgabeaakiabgkHiTiab=f7aHnaaDaaaleaacqWGPbqAaeaacqGGQaGkaaaakiaawIcacaGLPaaaaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabdYgaSbqdcqGHris5aOGaemiEaG3aaSbaaSqaaiabdMgaPbqabaGccqGH9aqpcqaIWaamcaWLjaGaaCzcaiabcIcaOiabikdaYiabikdaYiabcMcaPaaa@4ED5@
ξ i ( * ) L = C α i ( * ) η i ( * ) = 0 ( 23 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGHciITdaWgaaWcbaacciGae8NVdG3aaSbaaWqaaiabdMgaPbqabaaaleqaaOGaeiikaGIaeiOkaOIaeiykaKIaemitaWKaeyypa0Jaem4qamKaeyOeI0Iae8xSde2aa0baaSqaaiabdMgaPbqaaiabcIcaOiabcQcaQiabcMcaPaaakiabgkHiTiab=D7aOnaaDaaaleaacqWGPbqAaeaacqGGOaakcqGGQaGkcqGGPaqkaaGccqGH9aqpcqaIWaamcaWLjaGaaCzcaiabcIcaOiabikdaYiabiodaZiabcMcaPaaa@4B94@

From the substitution of (21), (22) and (23) into (19) we obtain a dual optimization problem.

Maximize  { 1 2 i , j = 1 l ( α i α i * ) ( α j α j * ) ( x i t x j ) ε i = 1 l ( α i α i * ) + i = 1 l y i ( α i α i * ) ( 24 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqqGnbqtcqqGHbqycqqG4baEcqqGPbqAcqqGTbqBcqqGPbqAcqqG6bGEcqqGLbqzcqqGGaaidaGabeqaauaabaqaceaaaeaacqGHsisldaWcaaqaaiabigdaXaqaaiabikdaYaaadaaeWbqaamaabmaabaacciGae8xSde2aaSbaaSqaaiabdMgaPbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaaaleaacqWGPbqAcqGGSaalcqWGQbGAcqGH9aqpcqaIXaqmaeaacqWGSbaBa0GaeyyeIuoakmaabmaabaGae8xSde2aaSbaaSqaaiabdQgaQbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemOAaOgabaGaeiOkaOcaaaGccaGLOaGaayzkaaWaaeWaaeaacqWG4baEdaqhaaWcbaGaemyAaKgabaGaemiDaqhaaOGaemiEaG3aaSbaaSqaaiabdQgaQbqabaaakiaawIcacaGLPaaaaeaacqGHsislcqWF1oqzdaaeWbqaamaabmaabaGae8xSde2aaSbaaSqaaiabdMgaPbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGSbaBa0GaeyyeIuoakiabgUcaRmaaqahabaGaemyEaK3aaSbaaSqaaiabdMgaPbqabaaabaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdGcdaqadaqaaiab=f7aHnaaBaaaleaacqWGPbqAaeqaaOGaeyOeI0Iae8xSde2aa0baaSqaaiabdMgaPbqaaiabcQcaQaaaaOGaayjkaiaawMcaaaaaaiaawUhaaiaaxMaacaWLjaGaeiikaGIaeGOmaiJaeGinaqJaeiykaKcaaa@8E7B@

Subject to i = 1 l ( α i α i * ) = 0 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaadaaeWbqaamaabmaabaacciGae8xSde2aaSbaaSqaaiabdMgaPbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaGaeyypa0JaeGimaadaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGSbaBa0GaeyyeIuoaaaa@3F49@ and α i , α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ [0, C]

Equation (22) can be rewritten as follows

w = i = 1 l ( α i α i * ) x i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWG3bWDcqGH9aqpdaaeWbqaamaabmaabaacciGae8xSde2aaSbaaSqaaiabdMgaPbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGSbaBa0GaeyyeIuoakiabdIha4naaBaaaleaacqWGPbqAaeqaaaaa@42DC@
(2)

, thus f ( x ) = i = 1 l ( α i α i * ) ( x t x ) + b MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGMbGzcqGGOaakcqWG4baEcqGGPaqkcqGH9aqpdaaeWbqaamaabmaabaacciGae8xSde2aaSbaaSqaaiabdMgaPbqabaGccqGHsislcqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaGccaGLOaGaayzkaaWaaeWaaeaacqWG4baEdaahaaWcbeqaaiabdsha0baakiabdIha4bGaayjkaiaawMcaaaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdGccqGHRaWkcqWGIbGyaaa@4B37@     (25)

This is the Support Vector expansion, i.e., the description of w as a linear combination of x i .

To compute b, it is necessary to use Karush-Kuhn-Tucker (KKT) conditions [45, 46]. These authors state that at the point of the solution the product between dual variables and constraints has to vanish.

α i (ε + ξ i - y i + (wtx i ) + b) = 0     (26)

α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@
(3)

(ε + ξ i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaH+oaEdaqhaaWcbaGaemyAaKgabaGaey4fIOcaaaaa@30E6@ + y i - (wtx i ) - b) = 0

and

(C - α i )ξ i = 0     (27)

(C - α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ ) ξ i MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqaH+oaEdaqhaaWcbaGaemyAaKgabaGaey4fIOcaaaaa@30E6@ = 0

From (26) and (27) it follows that:

  1. (i)

    Only samples (x i , y i ) with corresponding b L = i = 1 l ( α i * α i ) = 0 ( 21 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGHciITdaWgaaWcbaGaemOyaigabeaakiabdYeamjabg2da9maaqahabaWaaeWaaeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaOGaeyOeI0Iae8xSde2aaSbaaSqaaiabdMgaPbqabaaakiaawIcacaGLPaaacqGH9aqpcqaIWaamcaWLjaGaaCzcaiabcIcaOiabikdaYiabigdaXiabcMcaPaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaemiBaWganiabggHiLdaaaa@4931@ = C lie outside the ε-insensitive tube;

  2. (ii)

    α i α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ = 0

From (i) and (ii), it is possible to conclude that

ε - y i + (wtx i ) + b ≥ 0 and ξ i = 0 if α i <C     (28)

ε - y i + (wtx i ) + b ≤ 0     if α i > 0     (29)

In conjunction with an analogous analysis on α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@

max{-ε + y i - (wtx i )|α i <C or α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ > 0} ≤ b ≤ min{-ε + y i - (wtx i )|α i > 0 or α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ <C}     (30)

If some α i * MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacqWFXoqydaqhaaWcbaGaemyAaKgabaGaeiOkaOcaaaaa@30B6@ (0, C) the inequalities become equalities.

To point out the sparsity of the SV expansion: from (26), the Lagrange multipliers may be nonzero only for |f (x i ) - y i | ≥ ε.

Therefore, we have a sparse expansion of w in terms of x i [47].

Abbreviations

LR:

Loess Regression

SS:

Splines Smoothing

WS:

Wavelets Smoothing

KR:

Kernel Regression

SVR:

Support Vector Regression

References

  1. 1.

    Quackenbush J: Microarray data normalization and transformation. Nat Genet 2002, 32: 496 -501. 10.1038/ng1032

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    Cullane AC, Perriere G, Considine EC, Cotter TG, Higgins DG: Between-group analysis of microarray data. Bioinformatics 2002, 18: 1600–1608. 10.1093/bioinformatics/18.12.1600

    Article  Google Scholar 

  3. 3.

    Durbin BP, Hardin JS, Hawkins DM, Rocke DM: A variance-stabilizing transformation for gene-expression microarray data. Bioinformatics 2002, 18: S105–110.

    Article  PubMed  Google Scholar 

  4. 4.

    Kepler TB, Crosby L, Morgan KT: Normalization and analysis of DNA microarray data by self-consistency and local regression. Genome biol 2002, 3: RESEARCH0037. 10.1186/gb-2002-3-7-research0037

    PubMed Central  Article  PubMed  Google Scholar 

  5. 5.

    Yang IV, Chen E, Hasseman JP, Liang W, Frank BC, Wang S, Sharov V, Saeed AI, White J, Li J, Lee NH, Yeatman TJ, Quackenbush J: Within the fold: assessing differential expression measures and reproducibility in microarray assays. Genome Biol 2002, 3: research0062.

    PubMed Central  PubMed  Google Scholar 

  6. 6.

    Schadt EE, Li C, Ellis B, Wong WH: Feature extraction and normalization algorithms for high-density oligonucleotide gene expression array data. J Cell Biochem 2001, 37: 120–125. 10.1002/jcb.10073

    Article  Google Scholar 

  7. 7.

    Hill AA, Brown EL, Whitley MZ, Tucker-Kellogg G, Hunter CP, Slonim DK: Evaluation of normalization procedures for oligonucleotide array data based on spiked cRNA controls. Genome Biol 2001, 2: RESEARCH0055. 10.1186/gb-2001-2-12-research0055

    PubMed Central  CAS  Article  PubMed  Google Scholar 

  8. 8.

    Yang YH, Speed T: Design issues for cDNA microarray experiments. Nat Rev Genet 2002, 3: 579–588.

    CAS  PubMed  Google Scholar 

  9. 9.

    Perou CM: Show me the data! Nat Genet 2001, 29: 373. 10.1038/ng1201-373

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, Aach J, Ansorge W, Ball CA, Causton HC, Gaasterland T, Glenisson P, Holstege FC, Kim IF, Markowitz V, Matese JC, Parkinson H, Robinson A, Sarkans U, Schulze-Kremer S, Stewart J, Taylor R, Vilo J, Vingron M: Minimum information about microarray experiment (MIAME)-toward standards for microarray data. Nat Genet 2001, 29: 365–371. 10.1038/ng1201-365

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Yang YH, Dudoit S, Luu P, Lin DM, Peng V, Ngai J, Speed TP: Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation. Nucleic Acids Res 2002, 30: e15. 10.1093/nar/30.4.e15

    PubMed Central  Article  PubMed  Google Scholar 

  12. 12.

    Beheshti B, Braude I, Marrano P, Thorner P, Zielenska M, Squire JA: Chromosomal localization of DNA amplifications in neuroblastoma tumors using cDNA microarray comparative genomic hybridization. Neoplasia 2003, 5: 53–62.

    PubMed Central  CAS  Article  PubMed  Google Scholar 

  13. 13.

    Saeed AI, Sharov V, White J, Li J, Liang W, Bhagabati N, Braisted J, Klapa M, Currier T, Thiagarajan M, Sturn A, Snuffin M, Rezantsev A, Popov D, Ryltsov A, Kostukovich E, Borisovsky I, Liu Z, Vinsavich A, Trush V, Quackenbush J: TM4: a free, open-source system for microarray data management and analysis. Biotechniques 2003, 34: 374–378.

    CAS  PubMed  Google Scholar 

  14. 14.

    Baird D, Johnstone P, Wilson T: Normalization of microarray data using a spatial mixed model analysis which includes splines. Bioinformatics 2004, 17: 3196–205. 10.1093/bioinformatics/bth384

    Article  Google Scholar 

  15. 15.

    Workman C, Jensen LJ, Jarmer H, Berka R, Gautier L, Nielsen HB, Saxild HH, Nielsen C, Brunak S, Knudsen S: A new non-linear normalization method for reducing variability in DNA microarray experiments. Genome Biology 2002, 3(9):research0048.1–0048.16. 10.1186/gb-2002-3-9-research0048

    Article  Google Scholar 

  16. 16.

    Wang J, Ma JZ, Li MD: Normalization of cDNA microarray data using wavelet regressions. Combinatorial Chemistry & High Throughput Screening 9: 783–791.

  17. 17.

    Nadaraya EA: On estimating regression. Theory of probability and its applications 1964, 10: 186–190. 10.1137/1110024

    Article  Google Scholar 

  18. 18.

    Watson GS: Smooth regression analysis. Sankya A 1964, 26: 359–372.

    Google Scholar 

  19. 19.

    Vapnik VN: The Nature of Statistical Learning Theory. Springer 1995.

    Google Scholar 

  20. 20.

    Ramakrishnan R, Dorris D, Lublinsky A, Nguyen A, Domanus M, Prokhorova A, Gieser L, Touma E, Lockner R, Tata M, Zhu X, Patterson M, Shippy R, Sendera TJ, Mazumder A: An assessment of Motorola CodeLink™ microarray performance for gene expression profiling applications. Nucleic Acids Research 2002., 30:

    Google Scholar 

  21. 21.

    Balagurunathan Y, Dougherty ER, Chen Y, Bittner ML, Trent JM: Simulation of cDNA microarrays via a parameterized random signal model. Journal of Biomedical Optics 2002, 7(3):507–523. 10.1117/1.1486246

    CAS  Article  PubMed  Google Scholar 

  22. 22.

    Dias R: A review of non-parametric curve estimation methods with application to Econometrics. Economia 2002, 2: 31–75.

    Google Scholar 

  23. 23.

    Archambeau C: Probabilistic models in noisy environment – and their application to a visual prosthesis for the blind. PhD thesis. Universite catholique de Louvain, Applied Sciences Faculty; 2005.

    Google Scholar 

  24. 24.

    Fan J, Tam P, Vande WG, Ren Y: Normalization and analysis of cDNA microarrays using within-array replications applied to neuroblastoma cell response to a cytokine. PNAS 2004, 101: 1135–1140. 10.1073/pnas.0307557100

    PubMed Central  CAS  Article  PubMed  Google Scholar 

  25. 25.

    Fan J, Peng H, Huang T: Semilinear high-dimensional model for normalization of microarray data: a theoretical analysis and partial consistency. J Am Stat Assoc 2005, 100(471):781–813. 10.1198/016214504000001781

    CAS  Article  Google Scholar 

  26. 26.

    Huang J, Wang D, Zhang C: A two-way semi-linear model for normalization and analysis of cDNA microarray data. J Am Stat Assoc 2005, 100(471):814–829. 10.1198/016214504000002032

    CAS  Article  Google Scholar 

  27. 27.

    Wang D, Huang J, Xie H, Manzella L, Soares MB: A robust two-way semi-linear modelo for normalization of cDNA microarray data. BMC Bioinformatics 2005., 6(14):

    Google Scholar 

  28. 28.

    The R project for statistical computing[http://www.r-project.org]

  29. 29.

    Evaluating different methods of microarray data normalization[http://mariwork.iq.usp.br/normalization/]

  30. 30.

    Cleveland WS, Grosse E, Shyu WM: Local regression models. In Chapter 8 Statistical Models in S. Wadsworth & Brooks/Cole Edited by: Chambers JM, Hastie TJ. 1992.

    Google Scholar 

  31. 31.

    Schumaker LL: Spline functions basic theory. New York: John Wiley & Sons; 1981.

    Google Scholar 

  32. 32.

    Prenter PM: Splines and variational methods. New York: John Wiley & Sons; 1975.

    Google Scholar 

  33. 33.

    Meyer Y: Wavelets Algorithms and Applications. Philadelphia: SIAM; 1993.

    Google Scholar 

  34. 34.

    Chui CK: An introduction to wavelets. San Diego: Academic Press; 1992.

    Google Scholar 

  35. 35.

    Härdle W: Smoothing techniques with implementation. New York: Springer-Verlag; 1990.

    Google Scholar 

  36. 36.

    Donoho DL, Johnstone IM: Minimax estimation via wavelet shrinkage. Annals of Statistics 1998, 26: 879–921. 10.1214/aos/1024691081

    Article  Google Scholar 

  37. 37.

    Vapnik V, Lerner A: Pattern recognition using generalized portrait method. Automatic and Remote Control 1963, 24: 774–780.

    Google Scholar 

  38. 38.

    Vapnik V, Chervonenkis A: A note on one class of perceptrons. Automatics and Remote Control 1964, 25.

    Google Scholar 

  39. 39.

    Vapnik V, Chervonenkis A: Theory of pattern recognition. Moskow: Nauka; 1974.

    Google Scholar 

  40. 40.

    Hisanori K, Oshima T, Asai K: Extracting relations between promoter sequences and their strengths from microarray data. Bioinformatics 2004, 21: 1062–1068. 10.1093/bioinformatics/bti094

    Google Scholar 

  41. 41.

    Vapnik VN: Statistical Learning Theory. New York: Wiley; 1998.

    Google Scholar 

  42. 42.

    Mangasarian OL: Nonlinear Programming. New York: McGraw-Hill; 1969.

    Google Scholar 

  43. 43.

    McCormick GP: Nonlinear Programming Theory Algorithms and Applications. New York: John Wiley and Sons; 1983.

    Google Scholar 

  44. 44.

    Vanderbei RJ: An interior point code for quadratic programming. In Statistics and Operations Research. Princeton Univ., NJ; 1997.

    Google Scholar 

  45. 45.

    Karush W: Minima of functions of several variables with inequalities as side constraints. In Master thesis. University of Chicago Department of Mathematics; 1939.

    Google Scholar 

  46. 46.

    Kuhn HW, Tucher AW: Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probabilistics. Berkeley University of California Press; 1951:481–492.

    Google Scholar 

  47. 47.

    Smola AJ, Schölkopf B: A tutorial on support vector regression. Statistics and Computing 2004, 14: 199–222. 10.1023/B:STCO.0000035301.49549.88

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by FAPESP, CAPES, CNPq, FINEP and PRP-USP.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mari Cleide Sogayar.

Additional information

Authors' contributions

AF – has made substantial contributions to conception and design of the study, analysis and interpretation of data and has been involved in drafting of the manuscript.

JRS – has made substantial contributions to conception and design of the study, analysis and interpretation of data and has been involved in drafting of the manuscript.

LOR – acquisition of the benchmark data and has been involved in drafting parts of the manuscript.

CEF – has discussed the results and critically revised the manuscript for important intellectual content and has given the final approval of the version to be published.

MCS – has directed the work on differentially expressed genes using the CodeLink™ platform and critically revised the manuscript for important intellectual content and has given the final approval of the version to be published.

Authors’ original submitted files for images

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Fujita, A., Sato, J.R., Rodrigues, L.d.O. et al. Evaluating different methods of microarray data normalization. BMC Bioinformatics 7, 469 (2006). https://doi.org/10.1186/1471-2105-7-469

Download citation

Keywords

  • Support Vector Regression
  • Kernel Regression
  • Spline Smoothing
  • High Gene Expression
  • Sinusoid Shape