Volume 7 Supplement 4
Symposium of Computations in Bioinformatics and Bioscience (SCBB06)
SEQOPTICS: a protein sequence clustering system
 Yonghui Chen^{1}Email author,
 Kevin D Reilly^{1},
 Alan P Sprague^{1} and
 Zhijie Guan^{2}
DOI: 10.1186/147121057S4S10
© Chen et al; licensee BioMed Central Ltd 2006
Published: 12 December 2006
Abstract
Background
Protein sequence clustering has been widely used as a part of the analysis of protein structure and function. In most cases single linkage or graphbased clustering algorithms have been applied. OPTICS (Ordering Points To Identify the Clustering Structure) is an attractive approach due to its emphasis on visualization of results and support for interactive work, e.g., in choosing parameters. However, OPTICS has not been used, as far as we know, for protein sequence clustering.
Results
In this paper, a system of clustering proteins, SEQOPTICS (SEQuence clustering with OPTICS) is demonstrated. The system is implemented with SmithWaterman as protein distance measurement and OPTICS at its core to perform protein sequence clustering. SEQOPTICS is tested with four data sets from different data sources. Visualization of the sequence clustering structure is demonstrated as well.
Conclusion
The system was evaluated by comparison with other existing methods. Analysis of the results demonstrates that SEQOPTICS performs better based on some evaluation criteria including Jaccard coefficient, Precision, and Recall. It is a promising protein sequence clustering method with future possible improvement on parallel computing and other protein distance measurements.
Background
Extracting useful information from biological sequences is an emerging problem with the rapid growth of biological sequences databases. Among biological sequences, protein sequences are an especially interesting category since protein is functionally essential in life and its alphabet is large (20 amino acids). There are several wellknown protein databases: Pfam [1] is a collection of protein families and domains which contains multiple protein alignments of these families; National Center for Biotechnology Information (NCBI) [2] protein sequence database is an integrated, textbased search and retrieval system that is very often used in biological research; SwissProt [3] is a protein sequence database which strives to provide a high level of annotation, a high level of integration with other databases, and a minimal level of redundancy; The Protein Information Resource (PIR) [4] serves as an integrated public resource of functional annotation of protein data to support genomic/proteomic research and scientific discovery. These databases are often used as data sources for protein sequence clustering study. In this paper two data sets are from Pfam since Pfam is a semiautomatic protein family database, which aims to be comprehensive as well as accurate and may be used for clustering results evaluation. SwissProt and NCBI protein databases are also applied as data sources because they contain most protein sequences and are very popular in biological research.
As more protein sequences become available, protein structure and function can be better studied with more accuracy and efficiency. Among this research, one of the most important computational methods is sequence clustering [5, 6]. The goal of clustering protein sequences is to get a biologically meaningful partitioning. Clustering a large set of protein sequences offers several advantages: Proteins are usually grouped into families based on the sequence similarity clustering, which provides some clues about the general features of that family and evolutionary evidence of proteins; Clustering also helps to infer the biological function of a new sequence by its similarity to some functionknown sequences; Moreover, protein clustering can be used to facilitate protein 3dimensional structure discovery, which is very important for understanding protein's function. Recently developed clustering methods have been successful in clustering a large number of sequences simultaneously. ProClust [7] uses a graph based approach and considers multidomain sequences; SYSTER [8] overcomes the problem of an asymmetric distance matrix by computing a local pairwise alignment after performing a BLAST [9] search. GeneRage [10] is a fast method for clustering large protein data sets. ProtoMap [11] applies some more elaborate considerations. Among those protein sequence clustering methods, the simplest and most widely used category are based on hierarchical clustering algorithm (single linkage) [12]. It aggregates all the sequences linked by a level of similarity above a given threshold, so that within a cluster any sequence is linked to at least one other sequence. This approach may yield fairly good results, but often a majority of sequences are grouped into one single huge cluster resulting from a massive chain effect due to multidomain proteins. Blastclust program, one part of BLAST package from NCBI, is an example of single linkage protein sequence clustering http://www.bioinformatics.ubc.ca/resources/tools/?name=blastclust. Another category, graphbased clustering algorithms, are also commonly employed due to the clustering quality. BAG [13] is a sequence clustering algorithms based on graph theory and is web available at http://bio.informatics.indiana.edu/sunkim/BAG/.
OPTICS (Ordering Points To Identify the Clustering Structure) [14] is a densitybased clustering method and is popular because it orders the data into a densitybased clustering structure corresponding to a broad range of parameter settings. For densitybased methods, it is difficult to decide the input parameters that the algorithm is sensitive to. OPTICS is a good solution to densitybased cluster ordering. Although it does not produce clusters explicitly, OPTICS generates an augmented ordering of data sets representing its densitybased clustering structure, and this structure can be visualized. Since OPTICS does not limit cluster extraction to global parameters, it is possible to extract cluster information interactively as well as automatically. SEQOPTICS, a sequences clustering system based on OPTICS, is presented in this paper. For any protein sequences clustering method a suitable distance measure needs to be chosen. Some functionally related sequences share little or no discernible sequence similarity and detection of these relationships is difficult. The general practice to carry out protein sequence clustering is based on pairwise sequence similarity/dissimilarity computed by algorithms such as SmithWaterman [15]. Some other protein distance measurement such as BLAST [9], FASTA [16] are also very commonly taken in existing systems.
Evaluating clustering results quality is another important issue in clustering analysis. For twodimensional data, it is clear that one can plot the data and read the distribution to tell how good the clustering results are. But in high dimension data or sequence clustering, direct visualization is normally not feasible. In protein sequence clustering, a popular metric of clustering quality is based on how well the clusters identified by the clustering algorithm match the protein families defined in some database by biological experts [8]. Another method is to compare results of SEQOPTICS with results of some existing methods by using certain validation techniques [17]. Both evaluation techniques are conducted in this paper. In the following the SEQOPTICS clustering system is explained. Then SEQOPTICS is tested with several biological data sets. Visualization results of the clustering are presented. Moreover, the clustering results are analyzed according to the protein families identified by biologist and are also compared with those of two existing methods, blastclust and BAG. Results demonstrate that SEQOPTICS performs better in terms of clustering quality. Some future work needs to be done with the system includes system speedup and algorithm optimization.
Methods
Data Sets
Protein Sequences Data Sets
Data set  1  2  3  4 

From  Pfam (197)  Pfam (268)  NCBI (319)  SwissProt (295) 
Families  cytoB(75)  bac_globin(51)  cytoC(86)  GAPDH(122) 
GABAR(54)  IGA1(98)  GABAR(44)  casein kappa(62)  
bac_globin(51)  band3(119)  GAPDH(47)  globin (111)  
glucokinase(17)  GFAT(78)  
GPCR(64) 
Data set l (see Table 1) contains 197 protein sequences from four different families in Pfam database: 75 sequences of cytochrom_B561 (cytoB), 54 sequences of GABA Receptor (GABAR), 51 sequences of bac_globin, and 17 sequences of glucokinase. Data set 2 contains 268 sequences of three families of globin superfamily from Pfam database: bac_globin containing 51 sequences, IGA1 containing 98 sequences, and band_3_cytochrome (band3) containing 119 sequences. Data set 3 contains 319 sequences from five families in NCBI: 86 cytochrome C (cytoC) sequences, 44 GABAR sequences, 47 GAPDH sequences, 78 GFAT sequences, and 64 GPCR sequences. Data set 4 contains 295 sequences of three families from SwissProt database including: 122 GAPDHs, 62 casein kappas, and 111 globins. For each data set, protein sequences from different families are mixed and randomized to minimize the effect of predefined manual clustering.
Computing distance
Our approach, consonant with others, starts with a distance measure. When data sets are from different protein families, it is a common practice to use a normalized pairwise local alignment score by SmithWaterman dynamic programming algorithm. There are several parameters in SmithWaterman, for example, scoring matrix, open gap penalty and extending gap penalty. Various scoring matrices including BLOSUM50 and PAM250 have been tried. BLOSUM50, which is also used in FASTA [16], is used as default in this paper. The default open gap penalty taken is 12 and the extending gap penalty is 2. The similarity score between two protein sequences is then calculated by the following normalization formula:
where S(a, b) is the SmithWaterman local alignment score between two sequences a and b; S(a, a) is the similarity score of sequence a to itself; S(b, b) is the score of sequence b to itself; and SN(a, b) is the normalized score.
The distance between two protein sequences is defined as:
Distance(a, b) = 1  SN(a, b);
With this normalization, every distance score is between 0 and 1. If other scoring methods are used instead of SmithWaterman, the distance measure needs to be adjusted appropriately.
OPTICS clustering
Some preliminary remarks on OPTICS have been given in the introduction. Some definitions of the concepts used in OPTICS are described as follows: An object p is in the εneighborhood of q if the distance from p to q is less than ε; A core object has at least MinPts neighbors in its εneighborhood. The reachability distance of p is the smallest distance such that p is densityreachable from a core object o. A cluster is a set of densityconnected objects which is maximal with respect to densityreachability. A reachability plot is a bar chart that shows each object's reachability distance in the order the object was processed which demonstrates the cluster structure of data. The final clusters can be extracted by either εcutoff or steepness of the plot. For more detailed information about OPTICS algorithm, please refer the original paper [14].
SEQOPTICS is implemented with a distance measure of sequences based on SmithWaterman algorithm. The core OPTICS part was tested with the data sets from OPTICS authors. Two parameters need to be chosen, ε and MinPts. In this paper, since the distance between any two protein sequences is between 0 and 1, a single ε for all data set may be used, for example, ε is set as 0.99, which is slightly smaller than 1. The MinPts used here is 10 based on the measurement of the experimental data sets. For the whole protein database, ε can still use any value between 0.95 to 0.99, MinPts should be set as the average number of sequences in a family.
There are two main advantages to apply OPTICS in protein sequences clustering analysis: 1) OPTICS can find the local density region; 2) OPTICS produces an augmented ordering of the sequences representing its density based clustering structure and this ordering can be visualized, for example, in the reachability plot. The cluster ordering actually contains information about every cluster, i.e., OPTICS enables the extraction of not only "traditional" cluster information, but also intrinsic clustering structure.
Results and Discussion
SEQOPTICS is applied to cluster the experimental data sets. Visualization results are presented also. These provide some clues about clustering structure. The final densitybased cluster sets are defined from the ordering reachability distance. To evaluate the resulting clustering set's biological accuracy, we need to compare it to a "true" cluster set. However, there is no generally accepted "true" cluster set. All automatical protein clustering methods are based on "all against all" sequence comparison and real clusters need to be verified by biological expertise. Since it is impossible to have "real" clustering, the original database clusters are assumed to be "real" clusters. That is the way that most automatic protein clustering does. For example, all sequences from the glucokinase family of Pfam are considered in the same cluster.
Visualization of the cluster structure
For data set 1, there are five valleys in Figure 2: The first two valleys are composed of sequences from cytochrom_B562; The third valley consists of sequences from glucokinase; The fourth valley contains sequences from GABAR family; The fifth valley are sequences from bac_globin family. For data set 2, there are three valleys in Figure 3: The first one is composed of sequences from bac_globin; The second valley is composed of sequences from band3 family; The third valley contains only sequences from IGA1. For data set 3, there are six valleys in Figure 4: The first one and last one contain only cytoC sequences; The second valley contains only sequences from GABAR; The third valley contains sequences GAPDH; The fourth valley contains GPCR sequences; The fifth valley contains only GFAT. For data set 4, there are four main valleys in Figure 5: The first valley contains only casein kappa sequences; The second and third valley contain exclusively globins; the fourth valley is composed of GAPDHs.
Those figures shows that each valley contains exclusively one sequences family. Assume a new protein sequences is found: you may throw it into a pool of sequences and find which valley it is located in, then it is very possible that this new protein is from the same family as those in the same valley.
Extraction of the clusters
The final densitybased clusters are extracted by using a cutoff value. For example, in Figure 2, the cutoff value is set as 0.860 (shown as the line reachability distance = 0.860). Under this cutoff condition, each valley between two sequences with reachability distance higher than the cutoff is a cluster. The sequence starting a valley with reachability distance higher than the cutoff is also in the same cluster as rest sequences in the valley. Any sequence with reachability distance higher than the cutoff is noise if it does not start a valley. Therefore, in Figure 2, there are four clusters give the cutoff value 0.860, which is decided by experience. Similarly, there are four clusters in Figure 3 given cutoff 0.745, six clusters in Figure 4 given cutoff 0.860, three clusters in Figure 5 given cutoff 0.820.
Validation of the cluster set
To evaluate the resulting cluster sets with respect to its biological accuracy, the following problems need to be addressed:

There is no generally accepted "true" cluster set. That is to say, those "true" clusters are always "biased". However, if appropriate data source is chosen, then the "bias" can be limited.

There are some automatically generated cluster sets and some manually generated cluster sets. Those cluster sets are usually organized in "families", thus make the validation easier.
Automatically generated cluster sets are not necessarily biologically correct. They are normally based on allagainstall sequence comparisons. Pfam is an example of this category. Pfam is a large collection of common protein domains and families based on the UniProtKB/SwissProt Protein Knowledge base. Pfam seeds contain the seed alignments of the families and therefore are more accurate than general Pfam families. In this paper Pfam seeds are used for testing to reduce "bias" of "true" cluster.
NCBI is probably the most complete protein sequences database. UniProtKB/SwissProt provides a high level of annotation (such as the description of the function of a protein). SEQOPTICS data are extracted from NCBI and SwissProt since they are probably two most complete databases in biological research. Those extracted data are further manually preprocessed, i.e., those protein similar in annotation and sequences are selected so that "bias" is reduced.
As has been mentioned earlier in this paper, the original database clusters are considered as the "true" clusters against which the algorithm derived clusters are evaluated. Based on this assumption, several statistics metrics are used to evaluate the result.
According to Krause's PhD thesis [8], a cluster set of n data points from the experiment can be represented by the values in a triangular matrix M, where for i <j, M_{ ij }= 1, if and only if i and j are in the same cluster and M_{ ij }= 0 otherwise. If T is a matrix of "true" clusters, the two cluster sets ("true" and "experimental") can be compared based on the following numbers:

a is "true positive," i.e., the number of sequence pairs clustered together in both sets, which can be denned as: a = (i, j)M_{ ij }= 1 ∧ T_{ ij }= 1, i <j

b is "false negative," i.e., the number of sequence pairs clustered together in the true cluster set, but not in the clustering solution, defined as: b = (i, j)M_{ ij }= 0 ∧ T_{ ij }= 1, i <j

c is "false positive," i.e., the number of sequence pairs clustered in the current solution, but not in the true cluster set, defined as: c = (i, j)M_{ ij }= 1 ∧ T_{ ij }= 0, i <j

d is "true negative," i.e., the number of sequence pairs not clustered in either current solution or the true cluster set, defined as: d = (i, j)M_{ ij }= 0 ∧ T_{ ij }= 0, i <j
There are many validation techniques as cited in reference [17]. In this paper three parameters are applied based on the above definitions: Precision, Recall [18, 19], and Jaccard Coefficient [20].
Precision is defined as:
Recall is defined as:
Jaccard Coefficient is defined as:
Comparison of Clustering Results
Data Set  BLASTClust  BAG  SEQOPTICS  

S  P  R  S  P  R  S  P  R  
l(Pfam)  0.05  1.00  0.05  0.27  1.00  0.27  0.83  0.99  0.84 
2(Pfam)  0.04  1.00  0.04  0.20  1.00  0.20  0.85  0.98  0.87 
3(NCBI)  0.11  1.00  0.11  0.60  1.00  0.60  0.66  0.82  0.78 
4(SwissProt)  0.06  1.00  0.06  0.50  1.00  0.50  0.81  0.99  0.82 
Same data sets are tested with two other clustering methods, BLASTClust [9] and BAG [13], using default parameters of these methods. BAG is a graph based clustering method and graph clustering is used in some popular protein clustering methods including ProClust [7], SYSTERS [21]. BLASTClust is chosen because it is from NCBI BLAST package and is a widely used hierarchical sequence clustering method. The validation of our experiments is based on Jaccard Coefficient, Precision and Recall comparison values as shown in Table 2. Table 2 demonstrates that SEQOPTICS produces good results relative to each original cluster set in terms of Jaccard Coefficient. Every SEQOPTICS Jaccard Coefficient is higher than 0.65 and the highest being 0.85. It is also seen in the table that SEQOPTICS outperforms BAG and BLASTClust on all the data sets chosen on this criterion. The performance with BAG exceeds BLASTClust for the same reason. However, BAG and BLASTClust tend to give more clusters than the "true" clusters, explaining why the Precision of those two methods on all data sets are 1. Take Pfam1 as an example, SEQOPTICS gives 4 clusters, BAG results in 24 clusters and BLASTClust gives 121 clusters. Therefore, BAG and BLASTClust give high Precision values and low Recall value. But neither of these two performs well in terms of Recall. Overall, SEQOPTICS performs better than BAG and BLASTClust and seems a promising method in terms of both clustering quality coupled with its graphical representation of clustering structure.
Although manual cluster sets combined with biological experiment and the experts' information are the ultimate validation criterion, computerevaluation can be considered a tool at the disposal of experts in evaluating clustering results.
Conclusion
In this paper we described a prototyped system, SEQOPTICS, for protein sequences clustering as shown in Figure 1. A core portion(phase) of the system is based on OPTICS clustering and visualization method, which we believe is being used here for the first time for protein sequence clustering. Prior to this phase, it is necessary to compute the distance between (protein) sequences. A normalized SmithWaterman score is used in this paper to compute the required distance. The last system phase, Results Analysis, demonstrates adequacy of our approach for smallscale data and the usefulness of the cluster structure visualization. According to Ankerst [14], one good feature of OPTICS is that it does not limit oneself to a single set of global parameters. An augmented cluster ordering contains information equivalent to density based clusterings corresponding to a broad range of parameter settings; as such, the cluster ordering is a versatile base for both automatic and interactive cluster analysis. A second good feature lies in the visualization of the data set distribution. Depending on data set size, one can either represent the clusterordering graphically for small data sets, or, employ an alternate technique (appropriate) for large data sets. This paper demonstrates that in SEQOPTICS the visualization of cluster structure is meaningful. The time complexity of SmithWaterman is O(n^{2}l^{2}), where n is the number of sequences and l is the average length of the sequence. The time complexity of OPTICS is O(n^{2}) in the implementation. Therefore the total time complexity is O(n^{2}l^{2}). This is an expensive method if SmithWaterman is the only choice of the distance measure. Fortunately there are some other options for the distance between two protein sequences, such as BLAST or FASTA which will dramatically decrease the overall time complexity. SEQOPTICS has proved its value for handling small data sets (<1000 sequences) in this paper. If this system is applied to a large data set, such as a whole database, future improvements are necessary to make it more successful. The following directions are considered in future: 1) use some other distance measure for protein sequence distance, e.g., BLAST or FASTA; 2) apply parallel computing tools, for example, Message Passing Interface(MPI) for large data sets; 3) implement visualization techniques for large data sets; 4) consider incremental cluster ordering algorithms since protein databases are very frequently being updated.
Declarations
Acknowledgements
We appreciate reviewers of this paper for their valuable suggestions.
This article has been published as part of BMC Bioinformatics Volume 7, Supplement 4, 2006: Symposium of Computations in Bioinformatics and Bioscience (SCBB06). The full contents of the supplement are available online at http://www.biomedcentral.com/14712105/7?issue=S4.
Authors’ Affiliations
References
 Bateman A, Birney E, Cerruti L, Durbin R, Etwiller L, Eddy SR, GriffithsJones S, Howe KL, Marshall M, Sonnhammer ELL: The Pfam Protein Families Database. Nucl Acids Res 2002, 30: 276–280. [http://nar.oupjournals.org/cgi/content/abstract/30/1/276] 10.1093/nar/30.1.276PubMed CentralView ArticlePubMedGoogle Scholar
 Benson DA, KarschMizrachi I, Lipman DJ, Ostell J, Wheeler DL: GenBank: update. Nucl Acids Res 2004, 32(90001):D23–26. [http://nar.oupjournals.org/cgi/content/abstract/32/suppl:1/D23] 10.1093/nar/gkh045PubMed CentralView ArticlePubMedGoogle Scholar
 Bairoch A, Apweiler R: The SWISSPROT protein sequence database and its supplement TrEMBL in 2000. Nucl Acids Res 2000, 28: 45–48. [http://nar.oupjournals.org/cgi/content/abstract/28/1/45] 10.1093/nar/28.1.45PubMed CentralView ArticlePubMedGoogle Scholar
 Wu CH, Huang H, Arminski L, CastroAlvear J, Chen Y, Hu ZZ, Ledley RS, Lewis KG, Mewes HW, Orcutt BC, Suzek BE, Tsugita A, Vinayaka CR, Yeh LSL, Zhang J, Barker WC: The Protein Information Resource: an integrated public resource of functional annotation of proteins. Nucl Acids Res 2002, 30: 35–37. [http://nar.oupjournals.org/cgi/content/abstract/30/1/35] 10.1093/nar/30.1.35PubMed CentralView ArticlePubMedGoogle Scholar
 Kriventseva EV, Biswas M, Apweiler R: Clustering and analysis of protein families. Current Opinion in Structural Biology 2001, 11(3):334–339. [http://www.sciencedirect.com/science/article/B6VS6–4372YM1G/2/84f5cb8a038d7bd7f7a3abddObe1c2f2] 10.1016/S0959440X(00)002116View ArticlePubMedGoogle Scholar
 MohseniZadeh S, Brezellec P, Risler JL: ClusterC, an algorithm for the largescale clustering of protein sequences based on the extraction of maximal cliques. Comput Biol Chem 2004, 28: 211–218. [http://www.sciencedirect.com/science/article/B73G2–4CRY60H2/2/50f202126fa9421613cdb2fe5da055a6] 10.1016/j.compbiolchem.2004.03.002View ArticlePubMedGoogle Scholar
 Pipenbacher P, Schliep A, Schneckener S, Schonhuth A, Schomburg D, Schrader R: ProClust: improved clustering of protein sequences with an extended graphbased approach. Bioinformatics 2002, 18(90002):182S191. [http://bioinformatics.oupjournals.org/cgi/content/abstract/18/suppl_2/S182]View ArticleGoogle Scholar
 Krause A: Large Scale Clustering of Protein Sequences. PhD thesis. der Universitat Bielefeld; 2002.Google Scholar
 Altschul SF, Gish W, Miller W, Meyers EW, Lipman DJ: Basic Local Alignment Search Tool. Journal of Molecular Biology 1990, 215(3):403–410. [http:////www.sciencedirect.com/science/article/B6WK7–45PV640–2J/2/57a0a7cd25f342367d4f1daf09ac89e8] 10.1006/jmbi.1990.9999View ArticlePubMedGoogle Scholar
 Enright AJ, Ouzounis CA: GeneRAGE: a robust algorithm for sequence clustering and domain detection. Bioinformatics 2000, 16(5):451–457. [http://bioinformatics.oupjournals.org/cgi/content/abstract/16/5/451] 10.1093/bioinformatics/16.5.451View ArticlePubMedGoogle Scholar
 Yona G, Linial N, Linial M: ProtoMap: automatic classification of protein sequences and hierarchy of protein families. Nucl Acids Res 2000, 28: 49–55. [http://nar.oupjournals.org/cgi/content/abstract/28/1/49] 10.1093/nar/28.1.49PubMed CentralView ArticlePubMedGoogle Scholar
 Sibson R: SLINK: an optimally efficient algorithm for the singlelink cluster method. The Computer Journal 1973, 16: 30–34. 10.1093/comjnl/16.1.30View ArticleGoogle Scholar
 Kim S, Gopu A: BAG: A Graph Theoretic Sequence Clustering Algorithm. International Journal of Data Mining and Bioinformatics 2006., 1(2):
 Ankerst M, Breunig MM, Kriegel HP, Sander J: OPTICS: Ordering Points To Identify the Clustering Structure. In SIGMOD 1999, Proceedings ACM SIGMOD International Conference on Management of Data, June 1–3, 1999, Philadelphia, Pennsylvania, USA. Edited by: Delis A, Faloutsos C, Ghandeharizadeh S. ACM Press; 1999:49–60.Google Scholar
 Smith T, Waterman M: Identification of common molecular subsequences. J Mol Biol 1981, 147: 195–197. 10.1016/00222836(81)900875View ArticlePubMedGoogle Scholar
 Pearson WR, Lipman DJ: Improved Tools for Biological Sequence Analysis. Proc Natl Acad Sci USA 1988, 85: 2444–2448. 10.1073/pnas.85.8.2444PubMed CentralView ArticlePubMedGoogle Scholar
 Halkidi M, Batistakis Y, Vazirgiannis M: On Clustering Validation Techniques. Journal of Intelligent Information Systems 2001, 17(2–3):107–145. [http://citeseer.ist.psu.edu/article/halkidi01clustering.html] 10.1023/A:1012801612483View ArticleGoogle Scholar
 Amit B, Baldwin B: Algorithms for Scoring Coreference Chains.1998. [http://citeseer.csail.mit.edu/153897.html]Google Scholar
 Vilain M, Burger J, Aberdeen J, Connolly D, Hirschman L: A modeltheoretic coreference scoring scheme. In MUC6 '95: Proceedings of the 6th conference on Message understanding. Morristown, NJ, USA: Association for Computational Linguistics; 1995:45–52.View ArticleGoogle Scholar
 Jaccard A: Nouvelles recherches sur la distribution florale. Bull Soc Vaudoises Sci Nat 1908, 44: 223270.Google Scholar
 Krause A, Stoye J, Vingron M: The SYSTERS protein sequence cluster set. Nucl Acids Res 2000, 28: 270–272. [http://nar.oxfordjournals.org/cgi/content/abstract/28/1/270] 10.1093/nar/28.1.270PubMed CentralView ArticlePubMedGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.