Skip to main content
Fig. 5 | BMC Bioinformatics

Fig. 5

From: Multi-head attention-based U-Nets for predicting protein domain boundaries using 1D sequence features and 2D distance maps

Fig. 5

The multi-head attention mechanism. Here the \(L \times 2\) input is split into two L × 1 inputs. Each of these L × 1 inputs are fed into a scaled dot product attention head as the query, key and values. There are two attention heads, each of those yielding an output of L × 1 size. The attention head outputs are concatenated to form a tensor of the size of \(L \times 2\), which is passed through a linear layer. The matrix multiplication with a weight matrix is applied to the input in the linear layer to generate the activation for the softmax function to predict the probability of two classes (in a domain boundary or not). The final output is \(L \times 2\), where L is the sequence length

Back to article page

BMC Bioinformatics

ISSN: 1471-2105

Contact us