Closed-form dual perturb and combine for tree-based models
Geurts, Pierre[Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation >]
Wehenkel, Louis[Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes et modélisation >]
Proceedings of the International Conference on Machine Learning (ICML 2005)
22nd International Conference on Machine Learning
[en] machine learning ; optimisation
[en] This paper studies the aggregation of predictions made by tree-based models for several perturbed versions of the attribute vector of a test case. A closed-form approximation of this scheme combined with cross-validation to tune the level of perturbation is proposed. This yields soft-tree models in a parameter free way, and reserves their interpretability. Empirical evaluations, on classiﬁcation and regression problems, show that accuracy and bias/variance tradeoﬀ are improved signiﬁcantly at the price of an acceptable computational overhead. The method is further compared and combined with tree bagging.