Sensitivity of the evaluation to the search method S
. Two different search methods were used in Step 1 of the high-recall evaluation (n = 29 search methods tested). The panels show MAP and bpref agreement between these two runs. A stronger agreement is observed for bpref than for MAP (MAP/MAP correlation coefficient: 0.9540, bpref/bpref: 0.9740). These results indicate that the high-recall evaluation protocol produces performance measures which are marginally dependent on the choice of the S
method used to perform Step 1.