Dataset | msl | bs | lr | Layer | Head | Unit |
---|---|---|---|---|---|---|
BC2GM | 256 | 32 | 5e−5 | 2 | 12 | 64 |
BC4CHEMD | 256 | 32 | 3e−5 | 4 | 12 | 64 |
BC5CDR-chem | 256 | 32 | 3e−5 | 4 | 12 | 64 |
BC5CDR-disease | 256 | 32 | 3e−5 | 4 | 12 | 64 |
NCBI-disease | 256 | 32 | 5e−5 | 4 | 8 | 96 |
JNLPBA | 256 | 32 | 5e−5 | 2 | 8 | 96 |
LINNAEUS | 256 | 32 | 5e−5 | 4 | 12 | 64 |
Species-800 | 256 | 32 | 5e−5 | 2 | 12 | 64 |