Dynamically Route Hierarchical Structure Representation toAttentive Capsule for Text Classification
Abstract
Representation learning and feature aggregation
are usually the two key intermediate steps in
natural language processing. Despite deep neural
networks have shown strong performance in the
text classification task, they are unable to learn
adaptive structure features automatically and
lack of a method for fully utilizing the extracted features. In this paper, we propose a novel
architecture that dynamically routes hierarchical
structure feature to attentive capsule, named HAC.
Specifically, we first adopt intermediate information of a well-designed deep dilated CNN to form
hierarchical structure features. Different levels
of structure representations are corresponding to
various linguistic units such as word, phrase and
clause, respectively. Furthermore, we design a
capsule module using dynamic routing and equip it
with an attention mechanism. The attentive capsule
implements an effective aggregation strategy for
feature clustering and selection. Extensive results
on eleven benchmark datasets demonstrate that the
proposed model obtains competitive performance
against several state-of-the-art baselines. Our code
is available at https://github.com/zhengwsh/HAC.