Free energy estimation of short DNA duplex hybridizations

Background Estimation of DNA duplex hybridization free energy is widely used for predicting cross-hybridizations in DNA computing and microarray experiments. A number of software programs based on different methods and parametrizations are available for the theoretical estimation of duplex free energies. However, significant differences in free energy values are sometimes observed among estimations obtained with various methods, thus being difficult to decide what value is the accurate one. Results We present in this study a quantitative comparison of the similarities and differences among four published DNA/DNA duplex free energy calculation methods and an extended Nearest-Neighbour Model for perfect matches based on triplet interactions. The comparison was performed on a benchmark data set with 695 pairs of short oligos that we collected and manually curated from 29 publications. Sequence lengths range from 4 to 30 nucleotides and span a large GC-content percentage range. For perfect matches, we propose an extension of the Nearest-Neighbour Model that matches or exceeds the performance of the existing ones, both in terms of correlations and root mean squared errors. The proposed model was trained on experimental data with temperature, sodium and sequence concentration characteristics that span a wide range of values, thus conferring the model a higher power of generalization when used for free energy estimations of DNA duplexes under non-standard experimental conditions. Conclusions Based on our preliminary results, we conclude that no statistically significant differences exist among free energy approximations obtained with 4 publicly available and widely used programs, when benchmarked against a collection of 695 pairs of short oligos collected and curated by the authors of this work based on 29 publications. The extended Nearest-Neighbour Model based on triplet interactions presented in this work is capable of performing accurate estimations of free energies for perfect match duplexes under both standard and non-standard experimental conditions and may serve as a baseline for further developments in this area of research.

Conclusions: Based on our preliminary results, we conclude that no statistically significant differences exist among free energy approximations obtained with 4 publicly available and widely used programs, when benchmarked against a collection of 695 pairs of short oligos collected and curated by the authors of this work based on 29 publications. The extended Nearest-Neighbour Model based on triplet interactions presented in this work is capable of performing accurate estimations of free energies for perfect match duplexes under both standard and non-standard experimental conditions and may serve as a baseline for further developments in this area of research.

Background
Predicting the stability of a DNA duplex from base sequences is a well studied problem nowadays. Nevertheless, the accuracy of DNA duplex stability predictions largely varies with sequence length, base composition and experimental conditions. The Thermodynamic Nearest-Neighbour (TNN) Model [1] is a state-of-theart approach that is used to estimate the stability of a single or a pair of DNA (or RNA) molecules based on pairwise base interactions and structural conformations. A large collection of thermodynamic nearest-neighbour parameters were acquired by interpolation of results obtained from various experimental processes like NMR [2] and optical melting studies [1,3]. The accuracy of computing free energies for DNA duplexes is an important aspect for all prediction methods, considering their direct application for selecting, for example, microarray probes that perfectly hybridize with their complements within a pre-specified hybridization interval, while avoiding self-hybridization for each probe [4]. Here we select four widely used, publicly available computer programs that implement the TNN Model using large numbers of experimentally derived thermodynamic parameters, namely: the MultiRNAFold v2.0 package [5,6] with two sets of thermodynamic parameters, the Vienna Package v1.8.1 [7] and the UNAFold v3.5 package [8].
The MultiRNAFold package (including the PairFold program for duplexes) predicts the minimum free energy, suboptimal secondary structures and free energy changes of one, two, or several interacting nucleic acid sequences. The thermodynamic model for the thermodynamic stability of a joint secondary structure for two DNA or RNA molecules at a given temperature is performed similarly to that of a single molecule [9], except that an intermolecular initiation penalty is added. The PairFold algorithm uses dynamic programming to calculate minimum free energy secondary structures and runs in time cubic in the lengths of the input sequences (Θ(n 3 )). PairFold uses RNA thermodynamic parameters from the Turner Laboratory [10] and DNA thermodynamic parameters from the Mathews and SantaLucia laboratories [11,12].
The Vienna Package consists of a suite of computer programs and libraries for prediction of RNA and DNA secondary structures. Nucleic acid secondary structure prediction is done via free energy minimization using three dynamic programming algorithms for structure prediction: the minimum free energy algorithm of [13], which produces a single optimal structure, the partition function algorithm of [14], which calculates base pair probabilities in a thermodynamic ensemble, and the suboptimal folding algorithm of [15], which generates all suboptimal structures within a given energy range of the optimal energy. UNAFold, the acronym for "Unified Nucleic Acid Folding", is a software package for RNA and DNA folding and hybridization prediction. UNAFold folds singlestranded RNA or DNA, or two single DNA or RNA strands, by computing partition functions for various states of hybridization. The partition functions will then help to derive base pair probabilities and stochastic samples of foldings or hybridizations. The package provides various energy minimization methods, which compute minimum free energy hybridizations and suboptimal foldings.
All three packages use similar dynamic programming algorithms for prediction of minimum free energy (MFE) and suboptimal structures and for partition function calculations. For the purposes of our work (i.e., DNA duplex MFE secondary structure prediction and free energy of hybridization), the main differences lie in the thermodynamic parameters used (SantaLucia or Mathews), and in the features considered (for example, the Vienna Package does not consider special types of poly-C hairpin loops in their model, whereas the other two packages do). Thus our first goal is to quantify the impact of these differences on the accuracy of DNA duplex free energy approximations. Throughout the paper, we use a set of measures that reflect the degree of similarity of calculated and experimental secondary structures and free energies. Based on these measures we quantify the accuracy of the predictions of the aforementioned programs using a collection of 695 experimental DNA duplex data that we collected from 29 publications.
We also introduce in this work an extended Nearest-Neighbour Model for perfect matches based on triplet interactions, that can approximate free energies for DNA duplexes under a wide range of temperatures, sodium and sequence concentrations. The model is similar to the one introduced in 1999 by Owczarzy et al. [16], the main difference residing in the inclusion of only triplet interactions for our model, rather than a mixture of singlets, doublets and triplets for the other. Thus, our second goal is to show that such a model outperforms simpler models based on doublet interactions and produces more accurate free energy approximations for DNA duplex hybridizations occurring in non-standard experimental environments (for example for different sodium concentrations or at different temperatures).

Results and Discussion
In this work, we compare similarities and correlations of free energy values calculated using three publicly available packages, namely MultiRNAFold, UNAFold and Vienna Package and a Nearest-Neighbour (NN) Model for perfect matches based on triplet interactions. For this purpose, we collected and used a data set with 695 pairs of short DNA sequences and we investigated what method produces the closest value to the experimental free energy and under what circumstances. We acknowledge the fact that not all sequence lengths are equally represented in the benchmark data set simply due to their availability and thus our analysis may apply better to shorter sequences. The majority (91.37%) of experimental free energy calculations were obtained for perfect (0 mismatches) and almost perfect matches (1 mismatch), thus the current DNA parameter sets tend to have higher accuracy for close-to perfect match DNA duplexes. Another bias in the analysis may come from the fact that some authors have already tried to reconcile the existing differences in free energy model parameters [17,18] by optimizing sets of DNA parameters using the same sequences already present in the benchmark data set.
Comparison of absolute differences between experimental and estimated free energies (MFE_AD) We begin the presentation of our results by introducing a measure that provides insights into "worst" and "best" estimates for minimum free energies. Thus, the first comparison involves the absolute differences between experimental and estimated free energies (MFE_AD) among all the methods for model evaluation (column 3 in Table 1) and model prediction (column 3 in Table 2). In an ideal scenario, the estimated free energy would equal the experimentally inferred one, nevertheless in practice we would settle for a low absolute difference. In both scenarios, namely the evaluation of free energy estimates and the evaluation of secondary structure predictions, the largest maximal MFE_AD (18.4 kcal/mol in both) were obtained for the PairFold-Mathews method, while the minimal MFE_AD (13 kcal/mol for EVAL-FE and 11.88 kcal/mol for EVAL-SS) corresponds to the UNAFold method (see Methods for details). The average differences for the EVAL-FE methods range between 2.41 kcal/mol (UNAFold) and 3.16 kcal/mol (Vienna Package), while for the prediction methods the interval is slightly shifted towards zero. We also observed a similar improvement trend for MFE_AD standard deviations of EVAL-SS methods versus EVAL-FE methods, a phenomenon that can be explained by the intrinsic regression-based construction of the underlying DNA parameters used by each method.

Comparison of root mean squared errors (RMSE)
We measure the root mean squared error between experimentally determined and predicted free energies. In an ideal scenario where predicted values equal experimental values, the RMSE would be zero, thus the lower the RMSE value is, the closer the predicted values are to the experimental ones. Here, all methods produce comparably low RMSEs, the lowest EVAL-FE RMSE (3.876)  Tables 3 and 4).

Comparison of Pearson correlation coefficients (r)
A correlation coefficient is traditionally defined as a symmetric, scale-invariant measure of association between two random variables, which takes values between -1 and 1.  [19] and a few other minor deviations for the data collected from four additional publications [20][21][22][23] were consistently noticed for all free energy calculation methods (see Figures 1 and 2). The majority of the deviations (e.g. Doktycz et al. [19]) may come from potentially different free energy interpolation functions used in those studies. If we consider only perfect match data, the TNN-Triplets-PM Model (see Methods) is capable of estimating free energies that correlate better (r = 0.92) with experimental values (see Figure 3), than all the other methods, which show an average correlation coefficient r = 0.68. We notice also an improvement in the RMSE for the TNN-Triplets-PM Model, compared to the other programs. To ensure that this improvement is due to the triplet aspect of the model rather than other confounding factors, we created a TNN-Doublets-PM Model that has been trained and evaluated on the same perfect match data set. A detailed description of the training and evaluation procedure is provided in Tables 5 and 6. For the complete data set with perfect matches measured at various temperatures and buffer concentrations, Figures 4, 5, 6, 7, 8 and 9 show that our TNN-Triplets-PM Model consistently produces better correlations and RMSEs, when we run a random design experiment using 10 000 randomly selected subsets with 67% duplexes (228 perfect match duplexes) used for training and 33% duplexes (112 perfect match duplexes) used for testing. The same high correlations can be observed when running the TNN-Triplets-PM Model on perfect match duplex free energies measured at a temperature of 25°C and 1 M sodium concentration, while for perfect match free energies measured at 37°C and 1 M sodium concentration, the other models produce better but still comparable correlations (0.9) and RMSEs (0.7) with the TNN-Triplets-PM Model.
Comparison of secondary structure similarity indexes of experimental and predicted secondary structures (SSSI) The accuracy of secondary structure prediction for various methods can be evaluated by using the newly introduced measure described in equation 5. The SSSI measure simply calculates the percentage of correctly predicted secondary structure bonds corresponding to the positions in each secondary structure (corresponding to each sequence in the duplex) that match the position in the experimental secondary structure, normalized by the sum of sequence lengths. Comparable mean SSSI values were produced by all methods with a maximal value of 96.44% attained by PairFold-SantaLucia. The lowest value (95.21%) was obtained with Vienna Package (see column 6 in Table 2). All methods have large standard deviation for SSSI values, thus suggesting a wide sample distribution.

Comparison of SENS, PPV and F for predicted secondary structures
The analysis of the variation for sensitivities and F-measures with respect to sequence length and GC content percentages reveals a common pattern for all prediction methods. Mean sensitivities higher than 0.9 and mean Fmeasures higher than 0.95 were obtained for all methods and all sequence lengths with one exception. For sequences of length 10 a major drop in sensitivities and Fmeasures can be observed (see Figures 10 and 11). The main cause for the abrupt drop in sensitivities seem to apply mostly for sequences whose experimentally determined secondary structures contain two consecutive mismatches (collected from [23]), thus partially supporting the hypothesis that the prediction models under investigation seem to be optimized to produce better results for almost complementary pairs of DNA sequences. Next we look at how GC content % impacts the accuracy of prediction for the methods under consideration. While sensitivities and F-measures are higher than 0.9 for all methods for a wide range of GC content % intervals (e.g. 0% -10%, 40% -100%), there are values for which sensitivities and Fmeasures drop under 0.9 for sequences with GC content percentages in the range 10% -40%. While Pairfold-Mathews, Pairfold-SantaLucia and UNAFold generate predictions with sensitivities higher than 0.9 for sequences with GC content percentages in the range 20% -30%, the Vienna Package has a mean sensitivity of only 0.8. For 3 out of 4 methods, the PPV equals 1 (maximum), while for the remaining one, namely the Vienna Package slightly lower mean values (0.98) were obtained.
Comparison of free energy parameters for DNA doublets measured at 37°C and 1 M sodium concentration Table 2 presents the estimated free energy parameters for DNA doublets measured at 37°C. The set of 10 parameters corresponds to the best set obtained with the procedure explained in Table 6. We compared our set of NN free energy parameters at 37°C with eight other sets of parameters reported by SantaLucia [18], namely the sets obtained by Gotoh [24], Vologodskii    [25], Breslauer [26], Blake [27], Benight [28], SantaLucia [29], Sugimoto [30] and the Unified set [31]. Our set of NN thermodynamic doublet parameters summarized in Figure 12 differs from the unified parameters by less than 0.5 kcal/mol in 8 out of 10 cases. We also notice that our NN set follows in general the reported qualitative trend in order of decreasing stability: GC/CG = CG/GC > GG/CC > CA/GT = GT/CA = GA/CT = CT/ GA > AA/TT > AT/TA > TA/AT with one exception, namely GG/CC has a higher weight than GC/CG and CG/GC, an effect that could be caused by the low representation of the GG/CC doublets in the training set and by the absence of duplex initiation parameters in our model.

Conclusions
In this work we showed that no major differences exist among free energy estimations of short DNA duplex hybridization when comparing four publicly available programs that employ various sets of thermodynamic parameters.
Here we introduce a simplified TNN Model based on triplets interactions for perfect match hybridizations of DNA duplexes. The model is able to approximate free energies for DNA duplexes under various experimental conditions with higher accuracy and lower RMSEs compared to the four publicly available programs considered in this work. The improvement is more noticeable for DNA duplexes at non-standard experimental temperature conditions (for example at 25°C). This improvement obtained with the TNN Model based on triplets could be explained by the presence of a larger set of parameters consisting of 32 unique triplets (compared to only 10 unique doublets in the classical TNN Model) that better capture the impact of sequence components on the overall free energy of a DNA duplex. An alternative and potential complementary explanation of these improvements is the use of a wider variety of experimental data points in the thermodynamic parameter extrapolation process (the model training stage) compared to the smaller and less diverse data sets used in the other four programs. Nevertheless, we notice that additional experimental data employing longer and more diverse sequences is required in order to obtain a better approximation of free energies for DNA duplexes at other non-standard experimental conditions. Three extensions of the TNN-Triplets-PM Model might improve its performance, given that additional experimental data that covers a higher percentage of the parameters and experimental condition combinations is obtained experimentally: (i) the model can incorporate weighted additive terms that account for hybridization initialization, temperature, pH, sodium concentration or sequence concentrations; (ii) the model can incorporate symmetrical and asymmetrical internal loops, multibranch loops, dangling ends and hairpin rules similar to those already existent in the classical TNN Model; (iii) the model can also incorporate positional dependencies of triplets with respect to the 5' and 3' ends of the sequences.

Methods
The present study is divided into two major sections: • Evaluation of free energy estimates (EVAL-FE): a comparative assessment of free energies calculated for DNA duplexes using different methods when both the duplex sequence and the duplex experimental secondary structure are given.
• Evaluation of secondary structure predictions (EVAL-SS): an accuracy assessment of secondary structure predictions when only the duplex sequence is given and the secondary structure is predicted.

Data
The benchmark data set used in this work consists of 695 experimental free energies and secondary structures for DNA duplexes, including 340 perfect matches and 355 imperfect matches. We collected these data from 29 publications and we present its characteristics in Table 1. We must mention that a total of 42 DNA duplexes were removed from the original data set (with 737 DNA

Table 5 Model training
Require: A thermodynamic model T, an input set S with perfect match DNA duplexes.
Ensure: An optimal set of thermodynamic DNA parameters X for the input model duplexes -see Additional file 1) because the ctEnergy function from UNAFold failed to produce valid free energies, due to the lack of DNA parameters for mismatches. The removed data corresponds to 30 duplexes from [31], 4 duplexes from [32], 4 duplexes from [33], 2 duplexes from [34] and 2 duplexes from [35]. The lengths of DNA sequences in the data set range from 4 nucleotides [29] to 30 nucleotides [36], some of them (length 8 and 9) being over represented (see Figure 13). The GC-content (%) of the sequences in the benchmark data set (see Figure 14) cover the whole spectra from 0% to 100%, with a dominant peak at 50%. Sequence concentrations range from 17.5 × 10 -6 M in [33] to 10 -4 M in [20,21,31,37,38]. The sodium concentration varies from 0.1 M in [39] and [40] to 1 M in 20 out of 29 sources. The reported free energies were measured at reaction temperatures ranging between 24.85°C [33,41] and 50°C [32].

Free energy calculations
In this study, three publicly available packages were used to calculate and compare the free energies for pairs of short DNA sequences: MultiRNAFold (with Mathews and SantaLucia parameters), UNAFold and the Vienna Package. All packages implement the TNN Model based on base doublet parameters.
The basic free energy calculations implemented in MultiRNAFold and Vienna Package are performed according to the Gibbs equation: where G°is the free energy measured, H°is the enthalpy, T is the absolute temperature measured in degrees Kelvin and S°is the entropy.
For a general two-state transition process of the type A + B ⇌ AB at equilibrium, the free energy change is calculated as follows: where R is the gas constant (1.98717 cal/(mol K)), T is the absolute temperature, and k is the equilibrium constant.

Computational procedures
The two main sections of this study, namely the evaluation of free energy estimates and the evaluation of secondary structure predictions, employ computational procedures made available in the corresponding software packages. The evaluation of free energy estimates (EVAL-FE) includes the following procedures: • The function free energy pairfold (sequence1, sequence2, known structure) is provided by the Mul-tiRNAFold package to compute the free energy for two sequences when the known secondary structure is given. The pairfold wrapper has been slightly modified to accept as parameters: two sequences, the temperature, the set of parameters (Mathews or SantaLucia), the nucleic acid (DNA or RNA) and the type of hybridization (with or without intra-molecular interactions between nucleotides).
• The function RNAeval is provided by the Vienna Package to compute the free energy for two sequences when the known secondary structure is provided. We wrote a Python wrapper that calls this function with the following parameters: -T temperature, -P dna.par. The wrapper also pre-processes the sequence and structure input so to satisfy the interactivity requirements of the RNAeval function.
• The function ctEnergy is provided by the UNAFold Package to compute the free energy for two sequences when the known secondary structure is given. We wrote a Python wrapper that preprocesses the sequences and structures into a CTformatted input file and calls the function with the following parameters: -n DNA, -t temperature, -N sodium concentration.
The evaluation of secondary structure predictions (EVAL-SS) includes the following procedures: • The function pairfold mfe (sequence1, sequence2, output structure) is provided by the MultiRNAFold package to compute the minimum free energy secondary structure for two DNA sequences that fold into 'output structure'. The pairfold wrapper has been slightly modified as described above.  • The function RNAcofold is provided by the Vienna Package to predict the free energy secondary structure for two sequences. A wrapper has been created for this function to accommodate the input and the parameters for the interactive interface as described above.
• The script UNAFold.pl is provided by the UNA-Fold Package to predict the free energy secondary structure for two sequences. We wrote a Python wrapper that pre-processes the sequences and structures into a CT-formatted input file and calls the function with the same parameters as for the ctEnergy function.

The TNN-Triplets-PM Model
For the case when only free energies for perfect matches are evaluated, we explore an approach that extends the classical TNN Model by looking at base triplets. A similar approach was introduced in 1999 by [16]. Pearson coeff. (r) Figure 6 Box plots for Pearson correlations (r) corresponding to 197 perfect match duplex free energies measured at 25°C. The figure represents box plots for Pearson correlation coefficients for 197 perfect match duplex free energies measured at 25°C and a sodium concentration of 1 M. The doublet-and triplet-based models were executed 10 000 times on randomly selected subsets with 67% training data and 33% testing data.  Figure 7 Box plots for RMSEs corresponding to 197 perfect match duplex free energies measured at 25°C. The figure represents box plots for RMSEs for 197 perfect match duplex free energies measured at 25°C and a sodium concentration of 1 M. The doublet-and tripletbased models were executed 10 000 times on randomly selected subsets with 67% training data and 33% testing data.  Figure 8 Box plots for Pearson correlations (r) corresponding to 143 perfect match duplex free energies measured at 37°C. The figure represents box plots for Pearson correlation coefficients for 143 perfect match duplex free energies measured at 37°C and a sodium concentration of 1 M. The doublet-and triplet-based models were executed 10 000 times on randomly selected subsets with 67% training data and 33% testing data.   compare the impact of doublet versus triplet interactions on the accuracy of free energy estimations, the approach proposed in this paper relies solely on triplet interactions, while the one proposed by [16] uses a more complex cumulative approach that combines singlet, doublet and triplet interactions within the same model. Due to rotational identities, only 32 different nearest-neighbour interactions are possible (out of a total of 64) for any Watson-Crick triplet structure. These interactions are enumerated in Table 2 together with corresponding parametric values obtained via a least-mean squared optimization solution for equation 3.
where F is a N × 32 matrix of counts for all perfect match data points, X is a vector with 32 unknown triplet parameter values, and R is a vector with N free energy experimental values for perfect matches. We solve the following equation: These values were obtained by using an over determined system of N equations (3) and solving equation 4 with a least-mean squared optimization function (implemented in the backslash operator for matrices) available in Matlab 7.7. Here N takes the value 228 (67% of 340 perfect match free energies measured at 25°C and 37°C), 132 (67% from 197 perfect match free energies measured at 25°C), or 96 (67% of 145 perfect match free energies measured at 37°C). The system with N equations has been extrapolated by selecting from the initial data set only the free energy measurements for perfect match DNA duplexes and counting the frequency of triplets in each duplex. Thus, for each duplex, the sum of parametric values for each triplet multiplied with its counts equals the experimental free energy. While our model is very simple and currently does not take into consideration mismatches, internal loops, and dangling ends, its strength is given by its ability to estimate perfect match DNA duplex free energies for a wide range of sodium, sequence and target concentrations and temperatures. This strength is given by the presence of a large and mixed training data set that was used to extrapolate the nearest-neighbour (NN) parameters for both the doublet-and the triplet-based models.

Model training and testing
The training process for the TNN-Triplets-PM Model is summarized in Table 5. We first process the input set, which contains perfect match DNA sequences and their corresponding experimentally derived free energies. The processing consists of scanning each perfect match sequence from left to right by moving a window of size 3 nucleotides (or 2 for the doublets) and counting the Figure 14 The distribution of GC-content percentages for the complete data set. The distribution of GC-content percentages for 695 duplexes. The majority of sequences have a 50% GC-content while only a few sequences have either low (10%, 20%) or high (100%) GC-content percentages.
frequency of each of the 32 unique triplets. We record each frequency at corresponding positions (i, j) in matrix F and each experimentally derived free energy is recorded at position i in matrix R. Here i represents the number of the sequence in the set and j represents the number of the triplet (from 1 to 32), whose frequency is recorded. After matrices F and R have been populated, a solution for equation 4 is computed and the value of vector X containing free energy parameters for all the unique triplets is reported.
The evaluation process of the TNN-Triplets-PM Model is summarized in Table 6. The evaluation process is repeated 10 000 times in this work. Each iteration consists of the following steps. First the data set is divided uniformly at random in a training set, TrS consisting of 67% of the data and a testing set, TeS that contains the remaining 33%. Next, the training process described in Table 5 is used to extrapolate the first set of perfect match triplet parameters. The derived parameters are used next to compute the Pearson momentum correlation coefficients and the RMSEs for each DNA perfect match duplex from TeS. Each correlation coefficient and RMSE is recorded in corresponding vectors to be analyzed later. The complete coverage of the triplet space, i.e. all possible triplets during the generation of training and testing sets using a randomized mechanism is not ensured for some of the 10 000 sets mostly due to the presence of a few under-represented (less than 20 CCC/GGG) or over-represented (more than 180 GAC/CTG) triplets that characterize the data set with perfect matches (see Figures 15 and 16). Nevertheless, we noticed that the training sets that produced the best results cover completely the triplet space. The same coverage was observed for the doublets.

Comparative measures
We use a large number of measures of similarity between experimental and computed free energies. Some of these measures were previously used by [42] to compare melting temperatures obtained with different methods and by [6] to estimate model parameters for RNA secondary structure prediction. If not stated otherwise, all comparisons in this paper were done on a data set comprising 695 pairs of DNA sequences collected from 29 publications. The measures used in this study are grouped in two categories, namely: Measures that evaluate accuracy of free energy estimations The following measures are used for free energy estimations of the known structures, as well as free energy estimations of predicted structures.
• the root mean squared error (RMSE), Measures that evaluate accuracy of secondary structure predictions • the secondary structure similarity index of experimental and predicted secondary structures (SSSI) • the prediction sensitivity for secondary structures (SENS) • the positive predictive value for secondary structures (PPV) • the F-measure for predicted secondary structures (F) For MFE_AD, SSSI, SENS, PPV and F we report the minimum, the first quartile, the median, the mean, the third quartile, the maximum and the standard deviation.
We define the secondary structure similarity index (SSSI) for two equally long structures as follows:

SSSI SS s exp s calc SS s exp s calc len s exp len s exp
2 ) ) ⋅ 100 (5) where s1 exp , s2 exp are two equally long structures obtained experimentally, s1 calc , s2 calc are two equally long calculated structures, and SS(a, b) is the total number of identical characters at corresponding positions in both structures. SSSI represents the percentage of positions in which two structures agree.
Unlike similar measures that assign a +1 score for two identical base pairs in two duplex structures, SSSI assigns a +1 score for two base pairs that have either the start or the end positions identical. This mechanism allows the differentiation between duplex secondary structures that have either one (score +1) or both (score +2) bases in a base pair correctly predicted.
The sensitivity, positive predictive value and F-measure are defined as in [6], namely:

Computational infrastructure
The entire analysis of this study was done with R version 2.5.1, Perl 5.8.8 and Python 2.5. All computations were carried out on a Open SuSe 10.2 Linux (kernel version 2.6.18.2) machine equipped with a Pentium 4, 2.8 GHz processor with 1 GB of RAM.
Additional file 1: Data set in comma separated value. The file contains information representing the data set used in this work. The data is structured on 15 columns as follows: (col 1) first sequence of the duplex, (col 2) second sequence of the duplex, (col 3) unique duplex ID containing the first and last authors of the papers that have first published the data, (col 4) dot-parenthesis notation of the secondary structure representation for the first sequence, (col 5) dot-parenthesis notation of the secondary structure representation for the second sequence, (col 6) experimental free energy measurement, (col 7) measurement error for the free energy, (col 8) experimental entropy measurement, (col 9) measurement error for the entropy, (col 10) experimental enthalpy measurement, (col 11) measurement error for the enthalpy, (col 12) experimental temperature of hybridization, (col 13) concentration for self-complementary sequences, (col 14) concentration for non self-complementary sequences, (col 15) [N a] + concentration. Click here for file [ http://www.biomedcentral.com/content/supplementary/1471-2105-11-105-S1.CSV ]