Skip to main content

Table 2 Criteria for measuring agreement

From: Evaluation of a large-scale biomedical data annotation initiative

Agreement Type Description
Strict similarity Exactly the same variable value between annotators.
Semantic similarity There is lexical discordance, but the words match to the same concept. This subsumes hierarchical similarity.
Partial similarity Partial agreement, some degree of discordance.