Skip to main content

Table 1 Prominent options for choosing loss function and regularizer in feature extraction algorithms

From: Sparse Proteomics Analysis – a compressed sensing-based approach for feature selection and classification of high-dimensional proteomics mass spectrometry data

Name

Loss function (L)

Regularizer (R)

AIC/BIC

∥y−〈ω,x〉∥2

∥ω∥0

Lasso

∥y−〈ω,x〉∥2

∥ω∥1

Elastic Net

∥y−〈ω,x〉∥2

\(\| \omega \|^{2}_{2}\) + ∥ω∥1

Regularized Least Absolute

  

Deviations Regression

∥y−〈ω,x〉∥1

∥ω∥1

Classic SVM

max(0,1−y〈ω,x〉)a

\( \frac {1}{2} \| \omega \|^{2}_{2}\)

ℓ 1-SVM

max(0,1−y〈ω,x〉)a

\( \frac {1}{2} \| \omega \|_{1}\)

Logistic Regression

log(1+exp(−y〈ω,x〉))

\( \frac {1}{2} \| \omega \|_{1}\)

  1. *This is the so called Hinge loss
  2. The ℓ 1- and ℓ 2-norm of a vector z=(z 1,…,z d )∈ℝ d are defined by \(\|z\|_{1}=\sum _{j=1}^{d}|z_{i}|\) and \(\|z\|_{2}=(\sum _{j=1}^{d} |z_{i}|^{2})^{1/2}\), respectively. The “ ℓ 0-norm” ∥z∥0, simply counts the number of non-zero entries of z
Back to article page

BMC Bioinformatics

ISSN: 1471-2105

Contact us