Abstract. We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of
images. We use DFF to gain insight into a deep convolutional neural network’s learned features, where we detect hierarchical cluster structures in
feature space. This is visualized as heat maps, which highlight semantically matching regions across a set of images, revealing what the network
‘perceives’ as similar. DFF can also be used to perform co-segmentation
and co-localization, and we report state-of-the-art results on these tasks.