Boosting and ℓ1-Penalty Methods for High-dimensional Data with Some Applications in Genomics
- 1.6k Downloads
We consider Boosting and ℓ1-penalty (regularization) methods for prediction and model selection (feature selection) and discuss some relations among the approaches. While Boosting has been originally proposed in the machine learning community (Freund and Schapire (1996)), ℓ1-penalization has been developed in numerical analysis and statistics (Tibshirani (1996)). Both of the methods are attractive for very high-dimensional data: they are computationally feasible and statistically consistent (e.g. Bayes risk consistent) even when the number of covariates (predictor variables) p is much larger than sample size n and if the true underlying function (mechanism) is sparse: e.g. we allow for arbitrary polynomial growth p = pn = O(nγ) for any γ > 0. We demonstrate high-dimensional classification, regression and graphical modeling and outline examples from genomic applications.
KeywordsSupport Vector Machine Base Procedure Weak Learner Undirected Edge Diagonal Linear Discriminant Analysis
Unable to display preview. Download preview PDF.
- BÜHLMANN, P. (2004): Boosting for high-dimensional linear models. To appear in the Ann. Statist.Google Scholar
- FREUND, Y. and SCHAPIRE, R.E. (1996): Experiments with a new boosting algorithm. In: Machine Learning: Proc. Thirteenth International Conference. Morgan Kauffman, San Francisco, 148–156.Google Scholar
- MEINSHAUSEN, N. and BÜHLMANN, P. (2004): High-dimensional graphs and variable selection with the Lasso. To appear in the Ann. Statist.Google Scholar
- TUKEY, J.W. (1977): Exploratory data analysis. Addison-Wesley, Reading, MA.Google Scholar
- WILLE, A., ZIMMERMANN, P., VRANOVÁ, E., FÜRHOLZ, A., LAULE, O., BLEULER, S., HENNIG, L., PRELIĆ, A., VON ROHR, P., THIELE, L., ZITZLER, E., GRUISSEM, W. and BÜHLMANN, P. (2004): Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biology, 5(11) R92, 1–13.CrossRefGoogle Scholar