Prediction using step-wise L1, L2 regularization and feature selection for small data sets with large number of features

Table 1 Prediction results

rank	task I	task II	taskIII	task IV^a
first	0.677	0.735	0.237	-2.578 (0.593)
second	0.627	0.612	0.201	-2.560 (0.565)
third	0.615	0.455	0.154	-2.561 (0.472)
stage 1
λ ₁	0.05	0.05	0.08	0.1
predict	0.667	0.642	0.205	-2.573 (0.548)
features^b	50	43	56	41
stage 2
λ₂	0.1	0.01	0.3	0.2
predict	0.691	0.668	0.131	-2.574 (0.586)

^a Numbers in brackets are Spearman Rank Correlation Coefficients (SRCC) [29].
^b number of features after L1 regularization.
Prediction results of q² values, eq. (5), for all four CoEPrA regression tasks using a two-step optimization procedure. First three lines display the results of the three best predictions for the different CEoPrA tasks. Stage 1: only L1 regularization is used. All features are removed, where the corresponding parameters have absolute values smaller than 10^-8 after optimization, Stage 2: only L2 regularization is applied for all features remaining after stage 1. The regularization parameters λ₁ and λ₂ have been determined using 5 times a 10-fold cross validation procedure.

ISSN: 1471-2105