Abstract
In this paper we aim to answer questions based on images when provided with a dataset of question-answer pairs
for a number of images during training. A number of methods have focused on solving this problem by using image
based attention. This is done by focusing on a specific part
of the image while answering the question. Humans also
do so when solving this problem. However, the regions that
the previous systems focus on are not correlated with the
regions that humans focus on. The accuracy is limited due
to this drawback. In this paper, we propose to solve this
problem by using an exemplar based method. We obtain
one or more supporting and opposing exemplars to obtain
a differential attention region. This differential attention is
closer to human attention than other image based attention
methods. It also helps in obtaining improved accuracy when
answering questions. The method is evaluated on challenging benchmark datasets. We perform better than other image based attention methods and are competitive with other
state of the art methods that focus on both image and questions