Comprior: facilitating the implementation and automated benchmarking of prior knowledge-based feature selection approaches on gene expression data sets

Table 1 Functional comparison of existing benchmarking tools for analyzing gene expression data

Approach	Domain	Benchmarking pipeline parts			Standard approaches	Exten-sibility	Prior knowledge	Evaluation functionality
Approach	Domain	Flexible design	Automated execution	Result analysis	Standard approaches	Exten-sibility	Prior knowledge	Standard metrics	Runtime	Biological relevance	Cross-validation	Visualization
iCOBRA [30]	Ranking comparison, binary assignments, e.g. DEA			\(\bullet\)				\(\bullet\)				\(\bullet\)
ClassifyR [3]	Gene expression feature selection, classification	(\(\bullet\))	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)		\(\bullet\)			\(\bullet\)	\(\bullet\)
DaMiRseq [1]	Gene expression feature selection, classification		\(\bullet\)	\(\bullet\)	\(\bullet\)			\(\bullet\)			\(\bullet\)	\(\bullet\)
OmicsMarkeR [2]	Omics feature selection, classification	(\(\bullet\))	\(\bullet\)	\(\bullet\)	\(\bullet\)			\(\bullet\)			\(\bullet\)
Comprior	Gene expression feature selection, classification	(\(\bullet\))	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)
NormalyzerDE [31]	Gene expression normalization, DEA	(\(\bullet\))	\(\bullet\)	\(\bullet\)	\(\bullet\)	\(\bullet\)		\(\bullet\)				\(\bullet\)
Summarized-Benchmarks [32]	General purpose	\(\bullet\)	\(\bullet\)	\(\bullet\)		\(\bullet\)		\(\bullet\)				\(\bullet\)
CellBench [33]	General purpose	\(\bullet\)	\(\bullet\)	\(\bullet\)		\(\bullet\)			\(\bullet\)
pipeComp [34]	General purpose	\(\bullet\)	\(\bullet\)	\(\bullet\)		\(\bullet\)		\(\bullet\)	\(\bullet\)			\(\bullet\)

The tools cover different parts of the benchmarking process and generally focus on result assessment based on standard performance metrics, e.g. accuracy. While the general purpose tools provide highest flexibility regarding pipeline design and extensibility, they do not provide built-in standard approaches for benchmarking. On the contrary, tools providing built-in approaches are typically not extendable. Comprior provides both built-in approaches and extensibility and is furthermore the only tool that focusses on prior knowledge retrieval and result set assessment regarding biological relevance

ISSN: 1471-2105