Boosting for high-dimensional linear models

Bühlmann, Peter

Ann. Statist., Tome 34 (2006) no. 1, p. 559-583 / Harvested from Project Euclid

Résumé

We prove that boosting with the squared error loss, L₂Boosting, is consistent for very high-dimensional linear models, where the number of predictor variables is allowed to grow essentially as fast as O(exp(sample size)), assuming that the true underlying regression function is sparse in terms of the ℓ₁-norm of the regression coefficients. In the language of signal processing, this means consistency for de-noising using a strongly overcomplete dictionary if the underlying signal is sparse in terms of the ℓ₁-norm. We also propose here an AIC-based method for tuning, namely for choosing the number of boosting iterations. This makes L₂Boosting computationally attractive since it is not required to run the algorithm multiple times for cross-validation as commonly used so far. We demonstrate L₂Boosting for simulated data, in particular where the predictor dimension is large in comparison to sample size, and for a difficult tumor-classification problem with gene expression microarray data.

Détail

Publié le : 2006-04-14
Classification:  Binary classification,  gene expression,  Lasso,  matching pursuit,  overcomplete dictionary,  sparsity,  variable selection,  weak greedy algorithm,  62J05,  62J07,  49M15,  62P10,  68Q32