Skip to main content

Table 5 Root mean squared error for P

From: A fast least-squares algorithm for population inference

K N α AD LS FRAPPE Significance LSα
2 100 0.10 4.33 4.37 4.33 AD = FR = LS 4.36
2 100 0.50 5.13 5.17 5.14 AD = FR = LS 5.17
2 100 1.00 5.99 6.03 5.99 AD = FR = LS 6.03
2 100 2.00 7.24 7.28 7.29 AD = LS = FR 7.25
2 1000 0.10 1.37 1.42 1.38 AD < FR < LS 1.39
2 1000 0.50 1.62 1.65 1.63 AD = FR < LS 1.65
2 1000 1.00 1.90 1.93 1.92 AD < FR = LS 1.93
2 1000 2.00 2.52 2.58 2.82 AD = LS < FR 2.38
2 10000 0.10 0.46 0.57 0.46 AD < FR < LS 0.48
2 10000 0.50 0.52 0.56 0.53 AD < FR < LS 0.52
2 10000 1.00 0.60 0.61 0.62 AD < LS < FR 0.61
2 10000 2.00 0.81 0.87 1.14 AD < LS < FR 0.92
3 100 0.10 5.58 5.64 5.58 AD = FR = LS 5.62
3 100 0.50 7.37 7.42 7.38 AD = FR = LS 7.42
3 100 1.00 9.05 9.06 9.06 AD = FR = LS 9.06
3 100 2.00 11.36 11.33 11.39 LS = AD = FR 11.30
3 1000 0.10 1.78 1.87 1.78 AD = FR < LS 1.80
3 1000 0.50 2.35 2.40 2.35 AD = FR < LS 2.39
3 1000 1.00 2.97 3.00 3.01 AD < LS = FR 3.00
3 1000 2.00 4.11 4.14 4.41 AD = LS < FR 3.89
3 10000 0.10 0.61 0.82 0.62 AD < FR < LS 0.61
3 10000 0.50 0.78 0.84 0.78 AD = FR < LS 0.76
3 10000 1.00 0.93 0.95 0.98 AD < LS < FR 0.95
3 10000 2.00 1.35 1.36 1.82 AD = LS < FR 1.49
4 100 0.10 6.83 6.90 6.84 AD = FR = LS 6.87
4 100 0.50 9.61 9.63 9.62 AD = FR = LS 9.62
4 100 1.00 11.90 11.89 11.92 LS = AD = FR 11.89
4 100 2.00 14.94 14.89 15.01 LS = AD = FR 14.89
4 1000 0.10 2.16 2.28 2.16 AD = FR < LS 2.17
4 1000 0.50 3.10 3.15 3.11 AD = FR < LS 3.15
4 1000 1.00 4.04 4.06 4.08 AD < LS = FR 4.06
4 1000 2.00 5.61 5.62 5.88 AD = LS < FR 5.36
4 10000 0.10 0.76 1.02 0.77 AD = FR < LS 0.71
4 10000 0.50 1.04 1.11 1.04 AD = FR < LS 1.01
4 10000 1.00 1.28 1.30 1.33 AD < LS < FR 1.30
4 10000 2.00 1.87 1.87 2.36 AD = LS < FR 2.06
  1. ‘AD’ = Admixture with ε = MN×10-4, ‘LS1’ = Least-squares with ε = MN×10-4 and α = 1, ‘FR’ = FRAPPE with ε = 1. Bold values indicate significantly less error than those without bold. ‘<’ indicates significantly less at 4.6e-4 level, and ‘=’ indicates insignificant difference. ‘LSα’ = Least-squares with correct α provided only for reference.