Abstract

 Learning to classify new categories based on just one or a few examples is a long-standing challenge in modern computer vision. In this work, we propose a simple yet effective method for few-shot (and one-shot) object recognition. Our approach is based on a modified auto-encoder, denoted -encoder, that learns to synthesize new samples for an unseen category just by seeing few examples from it. The synthesized samples are then used to train a classifier. The proposed approach learns to both extract transferable intra-class deformations, or "deltas", between same-class pairs of training examples, and to apply those deltas to the few provided examples of a novel class (unseen during training) in order to efficiently synthesize samples from that new class. The proposed method improves the state-of-the-art of one-shot object-recognition and performs comparably in the few-shot case.* The authors have contributed equally to this workCorresponding author: Leonid Karlinsky (leonidka@il.ibm.com)Figure 1: Visualization of two-way one-shot classification trained on synthesized examples. Cor-rectly classified images are framed in magenta (Golden retriever) and yellow (African wild dog). Theonly two images seen at training time and used for sample synthesis are framed in blue. Note thenon-trivial relative arrangement of examples belonging to different classes handled successfully byour approach. The figure is plotted using t-SNE applied to VGG features. Best viewed in color.32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada.