- Methodology article
- Open Access
Kernel based methods for accelerated failure time model with ultra-high dimensional data
© Liu et al; licensee BioMed Central Ltd. 2010
Received: 17 August 2010
Accepted: 21 December 2010
Published: 21 December 2010
Most genomic data have ultra-high dimensions with more than 10,000 genes (probes). Regularization methods with L1 and L p penalty have been extensively studied in survival analysis with high-dimensional genomic data. However, when the sample size n ≪ m (the number of genes), directly identifying a small subset of genes from ultra-high (m > 10, 000) dimensional data is time-consuming and not computationally efficient. In current microarray analysis, what people really do is select a couple of thousands (or hundreds) of genes using univariate analysis or statistical tests, and then apply the LASSO-type penalty to further reduce the number of disease associated genes. This two-step procedure may introduce bias and inaccuracy and lead us to miss biologically important genes.
The accelerated failure time (AFT) model is a linear regression model and a useful alternative to the Cox model for survival analysis. In this paper, we propose a nonlinear kernel based AFT model and an efficient variable selection method with adaptive kernel ridge regression. Our proposed variable selection method is based on the kernel matrix and dual problem with a much smaller n × n matrix. It is very efficient when the number of unknown variables (genes) is much larger than the number of samples. Moreover, the primal variables are explicitly updated and the sparsity in the solution is exploited.
Our proposed methods can simultaneously identify survival associated prognostic factors and predict survival outcomes with ultra-high dimensional genomic data. We have demonstrated the performance of our methods with both simulation and real data. The proposed method performs superbly with limited computational studies.
Survival prediction and prognostic factor identification play a very important role in medical research. Survival data normally include the censoring variable that indicates whether some outcome under observation (like death or recurrence of a disease) has occurred within some specific follow-up time. The modeling procedures must take into account such censoring. It is even more difficult to develop a proper statistical learning method for survival prediction.
Several models for survival predictions have been proposed in statistical literature. The most popular one is the Cox proportional hazards model [1–3], in which model parameters are estimated with partial log likelihood maximization. Another one is the accelerate failure time (AFT) model [4–6]. AFT is linear regression model in which the response variable is the logarithm or a known monotone transformation of a failure (death) time. There are mainly two approaches in literature for fitting a AFT model. One is the the Buckley-James estimator which adjusts censored observations using the Kaplan Meier estimator [7, 8], and the other is a semiparametric estimation of AFT model with an unspecific error distribution [9–11]. However, the semi-parametric Bayesian approach based on complex MCMC procedures is computationally intensive and tends to have inaccurate results, and the Stute's weighted least squares (LS) estimator only implicitly accounts for the censored time. The model has not been widely used in practice due to the difficulties in computing the model parameters , and there is no nonlinear AFT model in the literature.
Kernel based methods such as support vector machines (SVM) have been extensively studied recently in the framework of classification and regression  in the area of pattern recognition and statistical learning. The concept of kernel formulated as an inner product in the feature space allows us to build nonlinear extensions of many linear models . It would have been a potential alternative if it were not for the complexity of censoring. Moreover, LASSO type penalty and its generalized versions have been proposed for gene (variable) selection with high dimensional genomic profiles with censored survival outcomes [15–18]. However, since the sample size n ≪ m (the number of variables), methods based the primary formulation with a huge m (m > 40, 000) are not efficient. Consequently, in current microarray analysis, what people really do is select a couple of thousands (or hundreds) of genes using filter-based methods (such as T-test) and then apply the LASSO-type penalty to further reduce the number of disease associated genes. This two-step procedure will lead to missing biologically important genes and introducing bias. The dual solution with kernel proposed in this article attempts to resolve these inadequacies by solving a much smaller n × n matrix.
In this paper, we propose a nonlinear kernel ridge regression for censored survival outcome prediction under the framework of AFT model. We also develop an efficient dual solution with adaptive kernel ridge regression for ultra-high dimensional genomic data analysis.
Unlike the weighted least square method, our model explicitly accounts for censoring. The proposed models are evaluated with simulation and real data and the prediction error of the test data.
Results and Discussion
Sample 12 -dimensional input data x with 100 training and test samples respectively from a multivariate normal distribution with mean zero and variance-covariance matrix Σ. The pairwise correlation between the i th and the j th input variables in Σ is r|i-j|and different correlations (r = 0.2, 0.4, 0.6, and 0.8) will be chosen to assess the performance of the proposed method.
Choose the model parameters w = [1, 1, 1, 1, 1, 1, -1, -1, -1, -1, -1, -1] T , and generate the event time from log T = w t x k + ε, where ε ~ N(0, σ2) and σ is determined by the signal to noise ratio (SNR = μ surv /σ). For instance, with the mean log survival time of 3, and SNR = 3 : 1, we have σ = 1. SNR = 3 : 1 is used in all of our simulations. Finally, k indicates the k th power of input variables, so the log survival time is associated with the input variables nonlinearly.
The censoring variables are generated as uniformly distributed and independent of the events. Letting d i = (rand + C)T i , the censoring status will be δ i = T i < d i . Different Cs give a different portion of censored data. Roughly 40% - 60% censored data are produced in our simulations.
Frequencies of Correctly Identified variables with Different Parameters Out of 100 Simulations
|w i | = 1
|w i | = 0.2
|w i | = 0.1
Model performance with Simulation Data and Different Parameter Values
|w i |s(λ*, p*)
Av. # of Vars
1 (0.01, 0.6)
0.2 (0.002, 0.6)
0.1 (0.001, 0.6)
Computational Time (in Seconds): AKRR vs LASSO
Diffuse Large B-cell Lymphoma Data
Genes Associated with Survival Time for DLBCL Data
ribonucleotide reductase M2 polypeptide
tumor rejection antigen (gp96) 1
bone morphogenetic protein 6
CD86 antigen (CD28 antigen ligand 2, B7-2 antigen)
membrane-spanning 4-domains, subfamily A, member 2
CD79A antigen (immunoglobulin-associated alpha)
baculoviral IAP repeat-containing 3
lymphoid-restricted membrane protein
mitogen-activated protein kinase 10
immunoglobulin kappa constant
small inducible cytokine subfamily A (Cys-Cys), member 13
interleukin 1 receptor, type I
matrix metalloproteinase 9
LIM domain only 2 (rhombotin-like 1)
myeloid cell nuclear differentiation antigen
heat shock 70 kD protein 1A
microsomal glutathione S-transferase 1
inter-alpha (globulin) inhibitor H4
platelet-derived growth factor receptor, alpha polypeptide
Follicular Lymphoma (FL) Data
Genes Associated with Survival Time for FL Data
chromosome 20 open reading frame 51
complement component 4A (Rodgers blood group)
family with sequence similarity 54, member B
aldehyde dehydrogenase 2 family (mitochondrial)
Myosin phosphatase Rho-interacting protein
YLP motif containing 1
transmembrane protein 14A
HIV-1 induced protein HIN-1
peroxisomal biogenesis factor 3
haptoglobin-related protein; haptoglobin
tripartite motif-containing 31
Thirteen probes with over 100 relevance counts are identified. Those 13 probes are corresponding to 11 known genes associated with lymphoma and related diseases. For instance, gene C4A localizes to the major histocompatibility complex (MHC) class III region on chromosome 6. It plays a central role in the activation of the classical pathway of the complement system. C4A anaphylatoxin is a mediator of local inflammatory process. It induces the contraction of smooth muscle, increases vascular permeability, and causes histamine release from mast cells and basophilic leukocytes. C4A is on the pathway of Systemic Lupus Erythematosus (SLE). Patients with SLE can increase the risk of certain cancers, including non-Hodgkin's lymphoma. We find that C4A is negatively associated with survival time according the estimated coefficient of C4A. ALDH2 is another well studied gene which is significantly associated with acetaldehyde-induced micronuclei and alcohol-induced facial flushing. Defects in ALDH2 are a cause of acute alcohol sensitivity and alcohol induced cancers. There are accumulating evidences that ALDH2-deficient individuals are at much higher risk of esophageal cancer and malignant lymphoma. Our study indicates that the up-regulated ALDH2 is positively associated with patient survival outcomes. Six other genes are also associated with different cancers including follicular lymphoma.
We proposed kernel based methods for nonlinear AFT model and variable selection for ultra-high dimensional data. Our evaluations with simulation and real data illustrate that the proposed methods can effectively reduce the dimension of the covariates with sound prediction accuracy. In many studies, both clinical and genomic data are available. Due to the ultra-high dimension in genomic data, directly applying LASSO based methods to genomic data is usually not feasible. Our proposed method provides an efficient solution for it. Kernel based nonparametric methods have been well studied in statistical learning, but there are not many studies for survival analysis. In this paper, we provide a basis for further explorations in this field.
where M(x i ) > log T i if δ i = 0 and M(x i ) = log T i if δ i = 1. Because there are both equality and inequality constraints in the model, new methods need to be developed.
Kernel Ridge Regression (KRR)
where K = (K(x i , x j )) n × n = (ϕ(x i ) t ϕ(x j )) n × n is a kernel matrix which can be defined explicitly and K(x i ,.) = ϕ(x i ) t Φ t is the i-th row of the kernel matrix. Popular kernels include:
Radial basis function (Gaussian) kernel:
Our kernel ridge regression algorithm based on the dual equation (DKRR) (6) is as follows:
The Dual Kernel Ridge Regression (DKRR) Algorithm
Given λ, training data , test data , and a small ε.
Calculate the kernel matrices and .
Center the kernels and the survival times: and , where 1 n : a vector with n 1's. and I n : an identity matrix, and . Let a0 = [0, ..., 0, 0] t , and j = 0
FOR i = 1 to n,
j = j + 1
IF |a j+1 - a j | < ε, BREAK.
a j = aj+1
This dual kernel ridge regression (DKRR)algorithm designed for a quadratic error function with linear constraints is a convex function with convex constraints. Theoretically this algorithm will always converge and global optimal solution is guaranteed irrelevant to initial value a0. In our computational experiments, the differences of the estimated parameters with different initial values are very small (less than 0.01 with the infinity norm).
Adaptive Kernel Ridge Regression
Adaptive Kernel Ridge Regression (AKRR) Algorithm
Given a λ, p ∈ (0.1], training data , and a small ε and η.
Initializing w = u = rand(m, 1), and a = [0, ..., 0] t
Setting u(u i == 0) = 10e - 5 and j = 1.
While |w - u| > ε
u = w,
FOR i = 1 to n,
j = j + 1
w(w < η) = 0
Unlike other LASSO based methods which seek to find optimal w directly, AKRR algorithm updates the m-dimensional w through updating a much smaller n-dimensional dual variable a. This method is computationally highly efficient when n ≪ m, which is common in genomic data. Although the proposed method is based on the dual problem, the primal variable w is explicitly updated in the computation. Theoretically AKRR algorithm will always converge to global optimal solution when p = 1 irrelevant to initial values of w, u, and a, as the error function is convex under L1 penalty, but only local optimal solution is guaranteed when p < 1. However, in our computational experiments with simulation and real data, even though we may have different optimal solutions with different initializations only when p ≤ 0.5, most selected features are still the same in different runs. AKRR always reach the same optimal solution in all of our experiments when p ≥ 0.6. One possible explanation is that the error function may still be near convex or convex almost everywhere when p is large. Therefore it may be possible that we enjoy both the oracle property with less bias and the global optimal solution with larger p (0.6 ≤ p < 1). Theoretical study for the near convex error function, however, is out of the scope of this paper. To prevent the results stick to a local optimal solution when p ≤ 0.5, we run AKRR 30 times and the best solution is chosen from the run with smallest test error. Even though AKRR does choose different variables with different p s, a small subset (≥ 5) of most important genes are always selected in our experiments. The model performance can be evaluated with cross-validation and the relative root mean squared error of the test data. There are two parameters p and λ for the adaptive kernel ridge regression (AKRR) algorithm. One efficient way is to set p = 0.1, 0.2, ..., 1 alternatively, and then search for the best λ through cross-validation. The range of can be determined by the path of the optimal solution. λmin = 0 and λmax is set to be the smallest value with all zero estimated parameters by multiple trials. We search the optimal λ from λ ∈ (0, 1] in this paper. Usually we have a larger λmax for p = 1, and smaller λmax when p is smaller.
We thank the Associate Editor and the two anonymous referees for their constructive comments, which improve this manuscript significantly. This work was partially supported by the 1R03CA133899 grant from the National Cancer Institute and by the National Science Foundation CCF-0729080 grant.
- Cox D: Regression models and life-tables (with discussion). Journal of Royal Statistical Society, Series B 1972, 34: 187–220.Google Scholar
- Gui J, Li H: Variable Selection via Non-concave Penalized Likelihood and its Oracle Properties. Journal of the American Statistical Association, Theory and Methods 2001, 96: 456.Google Scholar
- Van Houwelingen H, Bruinsma T, Hart A, Van't Veer L, Wessels L: Cross-validated Cox regression on microarray gene expression data. Stat Med 2006, 25: 3201–3216. 10.1002/sim.2353View ArticlePubMedGoogle Scholar
- Kalbfleisch J, Prentice R: The Statistical Analysis of Failure Time Data. New York: John Wiley; 1980.Google Scholar
- Wei L: The accelerated failure time model. a useful alternative to the Cox regression model in survival analysis. Statistics in Medicine 1992, 11: 1871–1879. 10.1002/sim.4780111409View ArticlePubMedGoogle Scholar
- Ying Z: A large sample study of rank estimation for censored regression data. Annals of Statistics 1993, 21: 76–99. 10.1214/aos/1176349016View ArticleGoogle Scholar
- Stute W, Wang J: The strong law under random censorship. Annals of Statistics 1993, 14: 1351–1365.Google Scholar
- Stute W: Distributional convergence under random censorship when covariables are present. Scandinavia Journal of Statistics 1996, 23: 461–471.Google Scholar
- Christensen R, Johnson W: Modelling accelerated failure time with a Dirichlet process. Biometrika 1988, 75: 693–704. 10.1093/biomet/75.4.693View ArticleGoogle Scholar
- Kuo L, Mallick B: Bayesian semiparametric inference for the accelerated failure time model. Canadian J Stat 1997, 25: 457–472. 10.2307/3315341View ArticleGoogle Scholar
- Bedrick E, Christensen R, Johnson W: Bayesian accelerated failure time analysis with application to veterinary epidemiology. Stat Med 2000, 19: 221–237. 10.1002/(SICI)1097-0258(20000130)19:2<221::AID-SIM328>3.0.CO;2-CView ArticlePubMedGoogle Scholar
- Jin Z, Lin D, Wei L, Ying Z: Rank-based inference for the accelerated failure time model. Biometrika 2003, 90: 341–353. 10.1093/biomet/90.2.341View ArticleGoogle Scholar
- Vapnik V: Statistical Learning Theory. New York: Wiley and Sons; 1998.Google Scholar
- Shawe-Taylor J, Cristianini N: Kernel Methods for Pattern Analysis. London: Cambridge University Press; 2004.View ArticleGoogle Scholar
- Ma S, Huang J: Additive risk survival model with microarray data. BMC Bioinformatics 2007, 8: 192. 10.1186/1471-2105-8-192View ArticlePubMedPubMed CentralGoogle Scholar
- Sha N, Tadesse M, Vannucci M: Bayesian variable selection for the analysis of microarray data with censored outcomes. Bioinformatics 2006, 22(18):2262–2268. 10.1093/bioinformatics/btl362View ArticlePubMedGoogle Scholar
- Liu Z, Gartenhaus R, Chen X, Howell C, Tan M: Survival Prediction and Gene Identification with Penalized Global AUC Maximization. Journal of Computational Biology 2009, 16(12):1661–1670. 10.1089/cmb.2008.0188View ArticlePubMedPubMed CentralGoogle Scholar
- Liu Z, Jiang F: Gene identification and survival prediction with Lp Cox regression and novel similarity measure. Int J Data Min Bioinform 2009, 3(4):398–408. 10.1504/IJDMB.2009.029203View ArticlePubMedGoogle Scholar
- Rosenwald A, Wright G, Chan W, Connors J, Campo E, Fisher R, Gascoyne R, Muller-Hermelink H, Smeland E, Giltnane J, Hurt E, Zhao H, Averett L, Yang L, Wilson W, Jaffe E, Simon R, Klausner R, Powell J, Duffey P, Longo D, Greiner T, Weisenburger DD, DBLJVJAJMELGAGTMTLM, Sanger WG, Ott G, Kvaloy S, Delabie J, Holte H, Krajci P, Stokke T, Staudt L: The use of molecular profiling to predict survival after themotheropy for diffuse large-B-cell lymphoma. The New England Journal of Medicine 2002, 346: 1937–1947. 10.1056/NEJMoa012914View ArticlePubMedGoogle Scholar
- Dave S, Wright G, Tan B, Rosenwald A, Gascoyne R, Chan W, Fisher R, Braziel R, Rimsza L, Grogan T, Miller T, LeBlanc M, Greiner T, Weisenburger D, Lynch J, Vose J, Armitage J, Smeland E, Kvaloy S, Holte H, Delabie J, Connors J, Lansdorp P, Ouyang Q, Lister T, Davies A, Norton A, Muller-Hermelink H, Ott G, Campo E, Montserrat E, Wilson W, Jaffe E, Simon R, Yang L, Powell J, Zhao H, Goldschmidt N, Chiorazzi M, Staudt L: Prediction of survival in follicular lymphoma based on molecular features of tumor-in filtrating immune cells. N Engl J Med 2004, 351(21):2159–2169. 10.1056/NEJMoa041869View ArticlePubMedGoogle Scholar
- Knight K, Fu W: Asymptotics for Lasso-type estimators. Annals of Statistics 2000, 28: 1356–1378. 10.1214/aos/1015957397View ArticleGoogle Scholar
- Fan J, Peng H: On Nonconcave Penalized Likelihood With Diverging Number of Parameters. The Annals of Statistics 2004, 32: 928–961. 10.1214/009053604000000256View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<url>http://creativecommons.org/licenses/by/2.0</url>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.