CBESW: Sequence Alignment on the Playstation 3

Background The exponential growth of available biological data has caused bioinformatics to be rapidly moving towards a data-intensive, computational science. As a result, the computational power needed by bioinformatics applications is growing exponentially as well. The recent emergence of accelerator technologies has made it possible to achieve an excellent improvement in execution time for many bioinformatics applications, compared to current general-purpose platforms. In this paper, we demonstrate how the PlayStation® 3, powered by the Cell Broadband Engine, can be used as a computational platform to accelerate the Smith-Waterman algorithm. Results For large datasets, our implementation on the PlayStation® 3 provides a significant improvement in running time compared to other implementations such as SSEARCH, Striped Smith-Waterman and CUDA. Our implementation achieves a peak performance of up to 3,646 MCUPS. Conclusion The results from our experiments demonstrate that the PlayStation® 3 console can be used as an efficient low cost computational platform for high performance sequence alignment applications.


Background
Sequence alignment is a popular bioinformatics application that determines the degree of similarity between nucleotide or amino acid sequences which is assumed to have same ancestral relationships. The optimal local alignment of a pair of sequences can be computed by the dynamic programming (DP) based Smith-Waterman (SW) algorithm [1]. However, this approach is expensive in terms of time and memory cost. Furthermore, the exponential growth of available biological data [2] means that the computational power needed is growing exponentially as well.
The recent emergence of accelerator technologies such as FPGAs, GPUs and specialized processors have made it possible to achieve an excellent improvement in execution time for many bioinformatics applications, compared to current general-purpose platforms. However, special-purpose hardware implementations such as FPGAs [3,4] tend to be very expensive and hard-to-program. Hence, they are not suitable for many users. Recent usage of easily accessible accelerator technologies to improve the search time of the SW algorithm include Intel SSE2 [5], GPU [6] and CUDA [7].
Farrar [5] exploits the SSE2 SIMD multimedia extension of general-purpose CPUs. His implementation utilizes vector registers, which are parallel to the query sequence and are accessed in a striped pattern. Similar to the implementation by Rognes [8], a query profile is calculated only once for each database search. However, Farrar's implementation allows moving the conditional calculation of the F-matrix outside the inner loop. Therefore, this implementation achieves a speed up of factor 2-8 over the previous SIMD implementations by Wozniak [9] and Rognes [8].
Liu et al. [10] first reported the implementation of the Smith-Waterman algorithm on graphics hardware. The SW algorithm is implemented using the streaming architecture of GPUs by reformulating it in terms of computer graphics primitives. The implementation relies on OpenGL, in which a conversion of the problem to the graphical domain is needed, as well as a reverse procedure to convert back the results. Although, it achieves a high efficiency, programming in OpenGL requires specialized skills. Therefore, Manavski [7] re-implemented the SW algorithm on a GPU with the recently released C-based CUDA programming environment. The implementation performs from 2 to 30 times faster than any other previous attempt available on commodity hardware.
In this paper, we demonstrate how the PlayStation ® 3 (PS3), a commodity hardware powered by the Cell Broadband Engine [11], can be used as a low cost computational platform to accelerate the Smith-Waterman algorithm. Our implementation is able to outperform both the striped method on an Intel Core 2 Duo as well as the CUDA-based GPU implementation on a GeForce 8800 GTX.

The Smith-Waterman Algorithm
The Smith-Waterman algorithm is used to determine the optimal local alignment between two nucleotide or protein sequences. The algorithm compares two sequences by computing the similarity score by means of dynamic programming (DP). Two elementary operations are used: substitution and insertion/deletion (also called a gap operation). The original algorithm was proposed by Smith and Waterman[1] with a complexity of O(m 2 n) and was improved by Gotoh [12] to run at O(mn).
Consider two strings S1 and S2 with length m and n, respectively. The Smith-Waterman algorithm computes the similarity value M(i, j) of two sequences ending at position i and j of the two sequences S1 and S2, respectively. For affine gap penalties, i.e. α ≠ β, the computation of M(i, j), for 1 ≤ i ≤ m, 1 ≤ j ≤ n, is given in the following equations 1-3: where sbt is a character substitution cost table, α is the cost of the initial gap; β is the cost of the following gaps. For linear gap penalties, i.e. α = β, the above recurrence relations can be simplified as shown in equations 4: Initialization values are given as the following: Each position of the matrix M is a similarity value. The two segments of S1 and S2 producing this value can be determined by a trace-back procedure. Figure 1 illustrates an example of computing the local alignment between two sequences PAWHEAE and HEA-GAWGHEE using the Smith-Waterman algorithm with the BLOSUM 50 scoring matrix [13]. The highest score in the matrix (+28) is the optimal score for the alignment. The trace-back procedure, shown in form of arrows, shows that the optimal local alignment is AW-HE and AWGHE.

Cell Broadband Engine Architecture
The Cell Broadband Engine [14] (Cell BE) is a recently introduced single-chip heterogeneous multi-core processor, which is developed by Sony, Toshiba and IBM. The Cell BE offers a unique assembly of thread-level and datalevel parallelization options. It is operating at the upper range of existing processor frequencies (3.2 GHz for current models) and is projected to run at more than 5 GHz in the near future. Several examples of bioinformatics applications that has been ported to the Cell BE architecture include Folding@Home [15], FASTA [16], Clus-talW [16] and RAxML [17].
The Cell BE combines an IBM PowerPC Processor Element (PPE) and eight Synergistic Processor Elements (SPEs) [11]. An integrated high-bandwidth bus called the Element Interconnect Bus (EIB) connects the processors and their ports to external memory and I/O devices. The block diagram of the Cell BE architecture is shown in Figure 2.
The PPE is a 64-bit Power Architecture core and contains a 64-bit general purpose register set (GPR), a 64-bit floating point register set (FPR), and a 128-bit Altivec register set. It is fully compliant with the 64-bit Power Architecture specification and can run 32-bit and 64-bit operating systems and applications. Each SPE is able to run its own individual application programs. Each SPE consists of a processor designed for streaming workloads, a local memory, and a globally coherent Direct Memory Access (DMA) engine. The EIB is a 4-ring structure, and can transmit 96 bytes per cycle, for a bandwidth of 204.8 Gigabytes/second. The EIB can support more than 100 outstanding DMA requests.
The most distinguishing feature of the Cell BE lies within the variety of the processors it has, i.e. the PPE and the SPEs. Heterogenous multi-core systems can lead to decreased performance if both the operating system and application are unaware of the heterogeneity. The PPE is designed to run the operating system and, in many cases, the top-level control thread of an application, while the SPEs is optimized for compute intensive applications, hence, providing the bulk of the application performance.

Cell BE Mapping
Our sequence alignment implementation [see Additional file 1, 2 and 3] uses affine gap penalties and utilizes the Sequence alignment of YPKIEAIY and MPKIIEAIYEN Figure 1 Sequence alignment of YPKIEAIY and MPKIIEAIYEN. An example of computing the local alignment between two sequences PAWHEAE and HEAGAWGHEE using the Smith-Waterman algorithm with the BLOSUM 50 scoring matrix [13]. The highest score in the matrix (+28) is the optimal score for the alignment. The trace-back procedure, shown in form of arrows, shows that the optimal local alignment is AW-HE and AWGHE.
Block diagram of the Cell BE Architecture Figure 2 Block diagram of the Cell BE Architecture. The Cell BE architecture consists of 1 PPE and 8 SPEs. However, in the Play Station ® 3, only 6 SPEs are available. 128-bit wide SIMD vector registers of the SPEs for optimization. The vectorization strategy is based on a columnbased approach [5,8]. It also employs a static load balancing strategy, which means that the work load is known at the start and distributed equally across the SPEs. The code is written in C together with the Cell BE SIMD Multimedia Extension Language intrinsics and SPU intrinsics for portability. DMA transfers and mailbox functions are used for communication purposes.
A list of SPU Low-Level Specific and Generic Intrinsics used in our vectorized implementation, divided into categories, is shown in  Figure 3 illustrates the mapping of different stages of SWbased protein sequence database scanning application onto the Cell BE. The PPE starts by reading the query and the database from the respective files and then pre-processes the query sequences such that they are suitable for vector operations. The pre-processed query sequence, together with some context data, is sent to each respective SPEs, which in turn will generate its own query profile. This process is done using DMA transfers, namely mfc_get and mfc_put. Given a database D consisting of |D| sequences and k SPEs. Each SPE aligns the query sequence to database sequences. Pseudocode of the mapping is illustrated in Figure 4. Scores obtained from those alignments are sorted locally in the SPEs and the b highest scores are sent to the PPE, where they are sorted once again to obtain the b overall highest scores.
Due to the fact that the SPEs only have 256 Kbytes of local memory, which have to store program code and data, memory allocation is crucial for the SPE. The current longest sequence in the Swiss-Prot database is 35,213 amino acids (accession number A2ASS6). In order to accommodate for longer protein sequence in the future, we allocate dynamic memory for the database sequences of up to 64,000 amino acids per sequence. Due to these limitations, the maximum query sequence length allowed for our implementation is limited to 852.

Query Profile
In order to calculate M(i, j) in the SW DP matrix, the value sbt(S 1 [i], S 2 [j]) needs to be added to M(i-1, j-1). To avoid performing this table lookup for each element in the DP matrix, Rognes [8] and Farrar [5] suggested calculating a query profile parallel to the query sequence beforehand.
Assuming that S 1 , S 2 ∈ Σ* and S 1 is the query sequence, the query profile is defined as a set P = {P x | x∈Σ} consisting of |Σ| numerical strings of length l 1 each, where l 1 = |S 1 |. Each string P x ∈ P consists of all substitution table values that are needed to compute a complete column j of the DP matrix for which S 2 [j] = x. Pre-computing the query profile greatly reduces the amount of substitution table lookup in the SW DP matrix computation, since |Σ| is usually much smaller than |S 2 |.
The query profile can be calculated in a straightforward sequential layout [8] or in a more complex striped layout [5], as shown in figure 5. The values in the query profile for sequential and striped layout are defined in equation 4 and 5, respectively: where p is the number of segments and t is the segment length.
In the striped layout, p corresponds to the number of elements that can be processed in a SIMD vector register (e.g. for 128-bit wide SIMD registers, p = 8 when using 16-bit precision). The length of each segment, t is defined in equation 6. t = L(l 1 + p -1)/pO Both approaches allow efficient vectorization on SSE2compatible processors using the corresponding SIMD instruction set. Using the pre-calculated query profile, the computation of the DP matrix can be performed in column-wise order. Due to the simplified dependency relationship and parallel loading of the vector scores from memory, fast DP matrix calculations can be achieved. The advantage of the striped layout compared to the sequential layout is that data dependencies between vector registers are moved outside the inner loop. For instance, when calculating vectors for the DP matrices H or F with the sequential layout, the last element in the previous vector has to be moved to the first element in the current vector. When using the striped query layout, this needs to be done just once in the outer loop when processing the next subject sequence character.

Saturation Arithmetic
The inner loop of the algorithm requires saturation arithmetic, namely saturated additions and saturated subtractions. The Cell BE lacks the saturation arithmetic support, leaving the tasks to be handled by software instead of direct hardware support. In order to tackle this problem, we introduced two new functions, namely spu_adds and spu_subs to handle saturated additions and saturated subtractions, respectively.

Results and discussion
In this section, we analyze the performance of our parallel algorithm for various query sequence lengths using sequences from Swiss-Prot database. where Pseudocode of the Cell BE mapping  Table 2 shows the performance of our parallel algorithm on the above mentioned datasets. By using 6 SPEs available in the PS3, our parallel algorithm reaches a peak performance of 3,646.48 MCUPS for a query sequence of length 852 (accession number O60341).
We have compared the performance of our CBESW implementation with other publicly available implementations of SW-based protein database scanning, namely SSEARCH [20], Striped Smith-Waterman [5] and CUDA [7]. The query sequences, as well as their respective Swiss Prot accession numbers, used in the different performance comparisons are shown in Table 3.
SSEARCH [20] is a SW implementation which is part of the FASTA [21] package. The SSEARCH performance is benchmarked on an Intel Core 2 Duo 2.4 GHz CPU with 1 GB RAM. Both execution cores were used in the experiment. As shown in Figure 6, for a query sequence of length 852 (accession number O60341), SSEARCH achieves a performance of 121.91 MCUPS. Thus, our implementation is over 30 times faster. Figure 7 shows the performance comparison between PS3 and striped SW. Striped SW is also benchmarked on an Intel Core 2 Duo 2.4 GHz CPU with 1 GB RAM. Both execution cores were used in the experiment. As can be seen from the figure, for query sequences with length > 255 amino acids, our PS3 implementation achieves a higher MCUPS performance compared to striped SW. The PS3 peak performance is 1.64 times faster than striped SW for the query sequence of length 852.
The performance comparison between the PS3 implementation and CUDA-SW on one Nvidia GeForce 8800GTX is shown in Figure 8. The CUDA implementation experiment was conducted with a GeForce 8800GTX 512 MB installed in a PC with a Dual-Core AMD Opteron 2210 1.8 GHz CPU, 2 GB RAM running Fedora 6. The substitution matrix used is BLOSUM50. As can be seen from the figure, our implementation achieves a better MCUPS performance. The PS3 peak performance is 3 times faster compared to the peak performance CUDA implementation on a single Nvidia GeForce 8800GTX.
The query profile layout Figure 5 The query profile layout. The query profile layout for (a) sequential method, (b) striped method.

Conclusion
In this paper, we have demonstrated that the PlayStation ® 3, powered by the Cell Broadband Engine, can be effectively used to accelerate a biological sequence alignment application. In order to derive an efficient mapping onto this type of heterogeneous multi-core architecture, we have utilized SIMD vectorization and parallel data partitioning and communication techniques.
Our implementation achieves a peak performance of 3,646.48 MCUPS for a query sequence of length 852. Hence, the peak performance of our implementation is 30.1 times and 1.64 times faster than SSEARCH and striped SW, on an Intel Core 2 Duo 2.4 GHz. The PS3 peak performance is also 3 times faster compared to the peak performance CUDA implementation on a single Nvidia  GeForce 8800GTX.
The very rapid growth of biological sequence databases demands even more powerful high-performance solutions in the near future. Hence, our results are especially encouraging since high performance computer architectures are developing towards heterogeneous multi-core systems.
Due to the 256 KB memory limitation of the SPE local store, the maximum query sequence length in our current implementation is 852. One of the limiting factors is that the size of the query profile grows with the length of the query sequence. Part of our future work is therefore to tackle this limitation. A promising approach is to align subject sequences against separate chunks of the query profile. The complete query profile only needs to be stored once in the main memory instead of the local store of the SPE. This frees up more memory space for the SPEs and thus allows longer query sequences. Given a query sequence of length l, the query profile can be divided into n chunks in which each chunks contains a query profile of size l/n. The respective SPEs can then align a part of the chunk of the query profile it has and get the next chunk from outside memory via concurrent DMA transfer. Performance comparison with the SSEARCH implementation Figure 6 Performance comparison with the SSEARCH implementation. Performance comparison between our CBESW implementation with SSEARCH, in terms of MCUPS. All queries were run against Swiss-Prot release 55.2. Nine query sequences with lengths of 63 to 852 amino acids were used.

Availability and requirements
Performance comparison with the Striped Smith-Waterman implementation Figure 7 Performance comparison with the Striped Smith-Waterman implementation. Performance comparison between our CBESW implementation with Striped Smith-Waterman, in terms of MCUPS. All queries were run against Swiss-Prot release 55.2. Nine query sequences with lengths of 63 to 852 amino acids were used.
Performance comparison with the CUDA implementation on a single Nvidia GeForce 8800GTX Figure 8 Performance comparison with the CUDA implementation on a single Nvidia GeForce 8800GTX. Performance comparison between our CBESW implementation with CUDA implementation on a single Nvidia GeForce 8800GTX, in terms of MCUPS. All queries were run against Swiss-Prot release 55.2. Seventeen query sequences with lengths of 63 to 852 amino acids were used. The scoring matrix used for the CUDA implementation was BLOSUM 50.