Abstract
Clarifying user needs is essential for existing task-oriented dialogue systems. However, in real-world applications, developers
can never guarantee that all possible user demands are taken into account in the design
phase. Consequently, existing systems will
break down when encountering unconsidered
user needs. To address this problem, we propose a novel incremental learning framework
to design task-oriented dialogue systems, or
for short Incremental Dialogue System (IDS),
without pre-defining the exhaustive list of user
needs. Specifically, we introduce an uncertainty estimation module to evaluate the con-
fidence of giving correct responses. If there is
high confidence, IDS will provide responses to
users. Otherwise, humans will be involved in
the dialogue process, and IDS can learn from
human intervention through an online learning
module. To evaluate our method, we propose
a new dataset which simulates unanticipated
user needs in the deployment stage. Experiments show that IDS is robust to unconsidered
user actions, and can update itself online by
smartly selecting only the most effective training data, and hence attains better performance
with less annotation cost