CBESW: Sequence Alignment on the Playstation 3

The Smith-Waterman Algorithm

The Smith-Waterman algorithm is used to determine the optimal local alignment between two nucleotide or protein sequences. The algorithm compares two sequences by computing the similarity score by means of dynamic programming (DP). Two elementary operations are used: substitution and insertion/deletion (also called a gap operation). The original algorithm was proposed by Smith and Waterman[1] with a complexity of O(m²n) and was improved by Gotoh[12] to run at O(mn).

Consider two strings S 1 and S 2 with length m and n, respectively. The Smith-Waterman algorithm computes the similarity value M(i, j) of two sequences ending at position i and j of the two sequences S 1 and S 2, respectively. For affine gap penalties, i.e. α ≠ β, the computation of M(i, j), for 1 ≤ i ≤ m, 1 ≤ j ≤ n, is given in the following equations 1–3:

M(i, j) = max{M(i - 1, j - 1) + sbt(S 1[i]), S 2[j], E(i, j), F(i, j), 0}, (1)

E(i, j) = max{M(i, j - 1) - α, E(i, j-1) - β}, (2)

F(i, j) = max{M(i - 1, j) - α, F(i - 1, j) - β}, (3)

where sbt is a character substitution cost table, α is the cost of the initial gap; β is the cost of the following gaps. For linear gap penalties, i.e. α = β, the above recurrence relations can be simplified as shown in equations 4:

M(i, j) = max{M(i - 1, j - 1) + sbt(S 1[i]), S 2[j], M(i, j - 1) - α, M(i - 1, j) - α} (4)

Initialization values are given as the following: for 0 ≤ i ≤ m, 0 ≤ j ≤ n, M(i, 0) = M(0, j) = E(i, 0) = F(0, j) = 0. Each position of the matrix M is a similarity value. The two segments of S 1 and S 2 producing this value can be determined by a trace-back procedure.

Figure 1 illustrates an example of computing the local alignment between two sequences PAWHEAE and HEAGAWGHEE using the Smith-Waterman algorithm with the BLOSUM 50 scoring matrix [13]. The highest score in the matrix (+28) is the optimal score for the alignment. The trace-back procedure, shown in form of arrows, shows that the optimal local alignment is AW- HE and AWGHE.

Cell Broadband Engine Architecture

The Cell Broadband Engine[14] (Cell BE) is a recently introduced single-chip heterogeneous multi-core processor, which is developed by Sony, Toshiba and IBM. The Cell BE offers a unique assembly of thread-level and data-level parallelization options. It is operating at the upper range of existing processor frequencies (3.2 GHz for current models) and is projected to run at more than 5 GHz in the near future. Several examples of bioinformatics applications that has been ported to the Cell BE architecture include Folding@Home[15], FASTA[16], ClustalW[16] and RAxML[17].

The Cell BE combines an IBM PowerPC Processor Element (PPE) and eight Synergistic Processor Elements (SPEs)[11]. An integrated high-bandwidth bus called the Element Interconnect Bus (EIB) connects the processors and their ports to external memory and I/O devices. The block diagram of the Cell BE architecture is shown in Figure 2.

The PPE is a 64-bit Power Architecture core and contains a 64-bit general purpose register set (GPR), a 64-bit floating point register set (FPR), and a 128-bit Altivec register set. It is fully compliant with the 64-bit Power Architecture specification and can run 32-bit and 64-bit operating systems and applications. Each SPE is able to run its own individual application programs. Each SPE consists of a processor designed for streaming workloads, a local memory, and a globally coherent Direct Memory Access (DMA) engine. The EIB is a 4-ring structure, and can transmit 96 bytes per cycle, for a bandwidth of 204.8 Gigabytes/second. The EIB can support more than 100 outstanding DMA requests.

The most distinguishing feature of the Cell BE lies within the variety of the processors it has, i.e. the PPE and the SPEs. Heterogenous multi-core systems can lead to decreased performance if both the operating system and application are unaware of the heterogeneity. The PPE is designed to run the operating system and, in many cases, the top-level control thread of an application, while the SPEs is optimized for compute intensive applications, hence, providing the bulk of the application performance.

The SPE can access RAM through direct memory access (DMA) requests. The DMA transfers are handled by the Memory Flow Controller (MFC). The MFC provides the interface, by means of the EIB, between the local storage of the SPE and main memory. The MFC supports DMA transfers as well as mailbox and signal-notification messaging between the SPE and the PPE and other devices. Data transferred between local storage and main memory must be 128-bit aligned. The size of each DMA transfer can be at most 16 KB. DMA-lists can be used for transferring large amounts of data (more than 16 KB). A list can have up to 2,048 DMA requests, each for up to 16 KB.

The PS3 uses the Cell Broadband Engine as its CPU, hence making it possible for users to create a high-powered computing environment for a fraction of the cost of a Cell Blade server. The PS3 utilizes seven of the eight SPEs, in which the eighth SPE is disabled to improve chip yields, i.e. chips do not have to be discarded if one of the SPEs is defective. Only six of the seven SPEs are accessible to developers as one is reserved by the operating system. The power requirement for the PS3 is 120 V AC, 60 Hz and the power consumption approximately 380 W. Generally available PS3's can be used for scientific high performance computing through installation of Linux (e.g. Red Hat or Yellow Dog). Programs can be developed the using freely available C-based Cell BE SDK [18]. At the time of this writing, the retail price of the PlayStation^® 3 is US$ 399 for 40 GB and US$480 for 60 GB, while the retail price of the Nvidia GeForce 8800GTX card is US$529, and a Dell Optiplex 745 with Intel Core 2 Duo 2.4 GHz processor is US$871. A QS20 Blade Server with two Cell BE chips has a retail price of US$18,995. Thus, the PS3 offers a good alternative to other accelerator technologies.

Cell BE Mapping

Our sequence alignment implementation [see Additional file 1, 2 and 3] uses affine gap penalties and utilizes the 128-bit wide SIMD vector registers of the SPEs for optimization. The vectorization strategy is based on a column-based approach[5, 8]. It also employs a static load balancing strategy, which means that the work load is known at the start and distributed equally across the SPEs. The code is written in C together with the Cell BE SIMD Multimedia Extension Language intrinsics and SPU intrinsics for portability. DMA transfers and mailbox functions are used for communication purposes.

Figure 3 illustrates the mapping of different stages of SW-based protein sequence database scanning application onto the Cell BE. The PPE starts by reading the query and the database from the respective files and then pre-processes the query sequences such that they are suitable for vector operations. The pre-processed query sequence, together with some context data, is sent to each respective SPEs, which in turn will generate its own query profile. This process is done using DMA transfers, namely mfc_get and mfc_put. Given a database D consisting of |D| sequences and k SPEs. Each SPE aligns the query sequence to $\frac{\left|D\right|}{k}$ database sequences. Pseudocode of the mapping is illustrated in Figure 4. Scores obtained from those alignments are sorted locally in the SPEs and the b highest scores are sent to the PPE, where they are sorted once again to obtain the b overall highest scores.

Due to the fact that the SPEs only have 256 Kbytes of local memory, which have to store program code and data, memory allocation is crucial for the SPE. The current longest sequence in the Swiss-Prot database is 35,213 amino acids (accession number A2ASS6). In order to accommodate for longer protein sequence in the future, we allocate dynamic memory for the database sequences of up to 64,000 amino acids per sequence. Due to these limitations, the maximum query sequence length allowed for our implementation is limited to 852.

Query Profile

In order to calculate M(i, j) in the SW DP matrix, the value sbt(S₁[i], S₂[j]) needs to be added to M(i-1, j-1). To avoid performing this table lookup for each element in the DP matrix, Rognes[8] and Farrar [5] suggested calculating a query profile parallel to the query sequence beforehand.

Assuming that S₁, S₂ ∈ Σ* and S₁ is the query sequence, the query profile is defined as a set P = {P _x | x∈Σ} consisting of |Σ| numerical strings of length l₁ each, where l₁ = |S₁|. Each string P _x ∈ P consists of all substitution table values that are needed to compute a complete column j of the DP matrix for which S₂[j] = x. Pre-computing the query profile greatly reduces the amount of substitution table lookup in the SW DP matrix computation, since |Σ| is usually much smaller than |S₂|.

The query profile can be calculated in a straightforward sequential layout [8] or in a more complex striped layout [5], as shown in figure 5. The values in the query profile for sequential and striped layout are defined in equation 4 and 5, respectively:

where p is the number of segments and t is the segment length.

In the striped layout, p corresponds to the number of elements that can be processed in a SIMD vector register (e.g. for 128-bit wide SIMD registers, p = 8 when using 16-bit precision). The length of each segment, t is defined in equation 6.

t = ⌈(l₁ + p - 1)/p⌉ (6)

Both approaches allow efficient vectorization on SSE2-compatible processors using the corresponding SIMD instruction set. Using the pre-calculated query profile, the computation of the DP matrix can be performed in column-wise order. Due to the simplified dependency relationship and parallel loading of the vector scores from memory, fast DP matrix calculations can be achieved. The advantage of the striped layout compared to the sequential layout is that data dependencies between vector registers are moved outside the inner loop. For instance, when calculating vectors for the DP matrices H or F with the sequential layout, the last element in the previous vector has to be moved to the first element in the current vector. When using the striped query layout, this needs to be done just once in the outer loop when processing the next subject sequence character.

Saturation Arithmetic

The inner loop of the algorithm requires saturation arithmetic, namely saturated additions and saturated subtractions. The Cell BE lacks the saturation arithmetic support, leaving the tasks to be handled by software instead of direct hardware support. In order to tackle this problem, we introduced two new functions, namely spu_adds and spu_subs to handle saturated additions and saturated subtractions, respectively.