资源论文Know What You Don’t Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories

Know What You Don’t Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories

2019-09-22 | |  131 |   100 |   0 0 0
Abstract Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories. Another strand of work in L&V aims at pragmatically informative rather than “correct” object descriptions, e.g. in reference games. We combine these lines of research and model zero-shot reference games, where a speaker needs to successfully refer to a novel object in an image. Inspired by models of “rational speech acts”, we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories. As a result of this reasoning, the generator produces fewer nouns and names of distractor categories as compared to a literal speaker. We show that this conversational strategy for dealing with novel objects often improves communicative success, in terms of resolution accuracy of an automatic listener

上一篇:Informative Image Captioning with External Sources of Information

下一篇:Label-Agnostic Sequence Labeling by Copying Nearest Neighbors

用户评价
全部评价

热门资源

  • Regularizing RNNs...

    Recently, caption generation with an encoder-de...

  • Deep Cross-media ...

    Cross-media retrieval is a research hotspot in ...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Supervised Descen...

    Many computer vision problems (e.

  • Learning Expressi...

    Facial expression is temporally dynamic event w...