Table 2 Comparison of existing BERTs
From: Deep learning with language models improves named entity recognition for PharmaCoNER
Model | Corpus combination | Vocabulary |
|---|---|---|
BERT(Cased) | Wiki+Books(Original) | BERT |
BERT(Uncased) | Wiki+Books(Original) | BERT |
NCBI BERT(+P,Uncased) | Original+PubMed | BERT |
NCBI BERT(+P+M,Uncased) | Original+PubMed+MIMIC-III | BERT |
Spanish BERT(Cased) | Original+Spanish Wikipedia+OPUS | Spanish BERT |
Spanish BERT(Uncased) | Original+Spanish Wikipedia+OPUS | Spanish BERT |
MultiBERT(Cased) | Multilingual Wikipedia | MultiBERT |
MultiBERT(Uncased) | Multilingual Wikipedia | MultiBERT |
SciBERT(BertVoc,Cased) | Original+Biomedical+Scientific | BERT |
SciBERT(BertVoc,Uncased) | Original+Biomedical+Scientific | BERT |
SciBERT(SciVob,Cased) | Original+Biomedical+Scientific | SciBERT |
SciBERT(SciVob,Uncased) | Original+Biomedical+Scientific | SciBERT |
BioBERTv1.0(+P,Cased) | Original+PubMed | BERT |
BioBERTv1.0(+PMC,Cased) | Original+PMC | BERT |
BioBERTv1.0(+P+PMC,Cased) | Original+PubMed+PMC | BERT |
BioBERTv1.1(+P,Cased) | Original+PubMed | BERT |