Abstract
This paper examines various unsupervised
pretraining objectives for learning dialog context representations. Two novel methods of
pretraining dialog context encoders are proposed, and a total of four methods are examined. Each pretraining objective is fine-tuned
and evaluated on a set of downstream dialog
tasks using the MultiWoz dataset and strong
performance improvement is observed. Further evaluation shows that our pretraining objectives result in not only better performance,
but also better convergence, models that are
less data hungry and have better domain
generalizability