Skip to main content

Table 1 Prominent options for choosing loss function and regularizer in feature extraction algorithms

From: Sparse Proteomics Analysis – a compressed sensing-based approach for feature selection and classification of high-dimensional proteomics mass spectrometry data

Name Loss function (L) Regularizer (R)
AIC/BIC y−〈ω,x2 ω0
Lasso y−〈ω,x2 ω1
Elastic Net y−〈ω,x2 \(\| \omega \|^{2}_{2}\) + ω1
Regularized Least Absolute   
Deviations Regression y−〈ω,x1 ω1
Classic SVM max(0,1−yω,x〉)a \( \frac {1}{2} \| \omega \|^{2}_{2}\)
1-SVM max(0,1−yω,x〉)a \( \frac {1}{2} \| \omega \|_{1}\)
Logistic Regression log(1+exp(−yω,x〉)) \( \frac {1}{2} \| \omega \|_{1}\)
  1. *This is the so called Hinge loss
  2. The 1- and 2-norm of a vector z=(z 1,…,z d ) d are defined by \(\|z\|_{1}=\sum _{j=1}^{d}|z_{i}|\) and \(\|z\|_{2}=(\sum _{j=1}^{d} |z_{i}|^{2})^{1/2}\), respectively. The “ 0-norm” z0, simply counts the number of non-zero entries of z