Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers

BMC Bioinformatics

Table 3 Exact (CD) and approximate critical values of differences in rank sums, at the familywise error rate of α=.05

k	n	max(d)	Unadjusted		Bonferroni-adjusted
					1 × N comparison			N × N comparison
			CD	CD _N	CD	CD _N	CD _M	CD	CD _N	CD _Q	CD χ ²
5	5	20	11	10	13	13	13	14	15	14	16
	10	40	15	14	18	18	18	20	20	20	22
	25	100	23	22	29	28	28	32	33	31	35
	50	200	32	31	40	40	39	45	45	44	49
	100	400	45	44	57	56	55	64	63	61	69
10	5	45	20	19	27	27	26	30	32	31	40
	10	90	27	27	38	38	37	44	45	43	56
	25	225	43	42	60	60	58	70	70	68	89
	50	450	60	60	85	84	82	99	99	96	125
	100	900	85	84	120	119	115	141	140	136	177
25	5	120	46	46	70	72	69	83	88	86	141
	10	240	65	65	100	102	98	121	124	121	199
	25	600	103	102	160	161	154	194	196	191	315
	50	1200	145	145	227	227	218	276	278	270	445
	100	2400	205	204	321	321	308	392	392	381	629
50	5	245	91	91	146	152	145	175	190	185	376
	10	490	128	128	210	215	205	258	268	261	531
	25	1225	203	203	337	339	323	417	423	412	840
	50	2450	287	286	478	479	457	595	599	582	1188
	100	4900	405	405	677	678	646	844	846	824	1680
100	5	495	180	180	304	320	302	368	406	395	1019
	10	990	255	255	441	452	427	548	573	559	1441
	25	2475	403	403	708	714	676	891	906	883	2278
	50	4950	569	569	1005	1010	955	1271	1281	1249	3221
	100	9900	805	805	1425	1427	1350	1805	1812	1766	4555

Note: The tabled values satisfy the relation P(D ≥ tabled value) <.05. For presentational purposes, the approximate critical differences were rounded up to the smallest integer that is not less than the calculated value. Italicized figures in the right-most column represent critical differences exceeding the maximum value of d, denoted max(d), implying that none of the rank sum differences is significant at the α=.05 level

ISSN: 1471-2105