Abstract
Gradient Boosted Decision Trees (GBDT) is a
widely used machine learning algorithm, which
obtains state-of-the-art results on many machine
learning tasks. In this paper we introduce a
method for obtaining better results, by augmenting the features in the dataset between the iterations of GBDT. We explore a number of augmentation methods: training an Artificial Neural Network
(ANN) and extracting features from it’s last hidden
layer (supervised), and rotating the feature-space
using unsupervised methods such as PCA or Random Projection (RP). These variations on GBDT
were tested on 20 classification tasks, on which all
of them outperformed GBDT and previous related
work