Privacy-preserving Stacking with Application to
Cross-organizational Diabetes Prediction
Abstract
To meet the standard of differential privacy, noise
is usually added into the original data, which
inevitably deteriorates the predicting performance
of subsequent learning algorithms. In this paper,
motivated by the success of improving predicting
performance by ensemble learning, we propose
to enhance privacy-preserving logistic regression
by stacking. We show that this can be done either by sample-based or feature-based partitioning.
However, we prove that when privacy-budgets are
the same, feature-based partitioning requires fewer
samples than sample-based one, and thus likely
has better empirical performance. As transfer
learning is difficult to be integrated with a differential privacy guarantee, we further combine the
proposed method with hypothesis transfer learning
to address the problem of learning across different
organizations. Finally, we not only demonstrate
the effectiveness of our method on two benchmark
data sets, i.e., MNIST and NEWS20, but also apply
it into a real application of cross-organizational
diabetes prediction from RUIJIN data set, where
privacy is of a significant concern