Table 2 3D-Jury receiver operating characteristic (ROC) analysis. ${\overline{tp}}_R$ – average number of true positive (tp) models in the [0 – 10] false positive (fp) range, using the reliability score provided by the server as the discrimination threshold; ${\overline{tp}}_J$ – average number of tp in the [0 – 10] fp range using 3D-Jury score as the discrimination threshold; J₀ – lowest 3D-Jury score before observing the first bad model; $t{p}_{J_0}$ – number of good models at or above J₀ score; N _t – number of targets The table shows results for the on-line default version of 3D-Jury: 3J_1,A. Servers are ordered by ${\overline{tp}}_J$ descending. Missing ${\overline{tp}}_R$ values indicate servers that did not return reliability scores. Five servers with ${\overline{tp}}_R>{\overline{tp}}_J$ are shown in italics. In order to assess 3D-Jury scores (Jscore) as reliability scores, we performed a ROC analysis adapted for CASP and Livebench data, comparing Jscore to the reliability scores provided by the servers. In terms of the average number of true positive models ($\overline{tp}$), the 3D-Jury score exceeds the original server score in 27 cases, it falls short of it in 5 cases out of the 38 analysed.

From: Evaluation of 3D-Jury on CASP7 models