Fig. 4

The illustration of the multi-head self-attention mechanism. The multi-head self-attention mechanism operates on each amino acid node. \(m_{i}\left( q_{i}, k_{i}, v_{i}\right)\) represents the query vector, key vector and value vector of amino acid node \(m_{i}\). \(m_{i}\left( q_{i, h}, k_{i, h}, v_{i, j}\right)\) represents the query vector, key vector, and value vector of the hth type of amino acid node \(m_{i}\). This symbol \(\oplus\) indicates addition by element-wise