Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with
Multi-Level Vocabularies for Response Generation
Abstract
We study the task of response generation.
Conventional methods employ a fixed vocabulary and one-pass decoding, which not only
make them prone to safe and general responses but also lack further refining to the first generated raw sequence. To tackle the above two problems, we present a Vocabulary Pyramid Network (VPN) which is able to incorporate multi-pass encoding and decoding with
multi-level vocabularies into response generation. Specifically, the dialogue input and output are represented by multi-level vocabularies which are obtained from hierarchical clustering of raw words. Then, multi-pass encoding and decoding are conducted on the multilevel vocabularies. Since VPN is able to leverage rich encoding and decoding information
with multi-level vocabularies, it has the potential to generate better responses. Experiments on English Twitter and Chinese Weibo datasets demonstrate that VPN remarkably
outperforms strong baselines