From: SparkGC: Spark based genome compression for large collections of genomes
Chromosome | Stage | 1 worker node | 2 worker nodes | 3 worker nodes | 4 worker nodes | ||||
---|---|---|---|---|---|---|---|---|---|
Time (s) | % | Time (s) | % | Time (s) | % | Time (s) | % | ||
Chr1 | Pre-processing | 112 | 1.79 | 116 | 3.84 | 124 | 5.27 | 126 | 6.49 |
First-order | 5577 | 89.23 | 2618 | 86.72 | 2007 | 85.37 | 1648 | 84.81 | |
Second-order | 531 | 8.50 | 254 | 8.41 | 188 | 8.00 | 136 | 7.00 | |
Post-processing | 30 | 0.48 | 31 | 1.03 | 32 | 1.36 | 32 | 1.70 | |
Total | 6250 | 100 | 3019 | 100 | 2351 | 100 | 1943 | 100 | |
Chr13 | Pre-processing | 62 | 3.58 | 70 | 6.42 | 70 | 9.06 | 70 | 10.74 |
First-order | 1520 | 87.76 | 921 | 84.50 | 625 | 80.85 | 512 | 78.53 | |
Second-order | 120 | 6.93 | 69 | 6.33 | 47 | 6.08 | 39 | 5.98 | |
Post-processing | 30 | 1.73 | 30 | 2.75 | 31 | 4.01 | 31 | 4.75 | |
Total | 1732 | 100 | 1090 | 100 | 773 | 100 | 652 | 100 |