Abstract

Undoing the image formation process and therefore de-composing appearance into its intrinsic properties is a chal-lenging task due to the under-constrained nature of thisinverse problem. While significant progress has been madeon inferring shape, materials and illumination from imagesonly, progress in an unconstrained setting is still limited.We propose a convolutional neural architecture to estimatereflectance maps of specular materials in natural lightingconditions. We achieve this in an end-to-end learning formu-lation that directly predicts a reflectance map from the imageitself. We show how to improve estimates by facilitating additional supervision in an indirect scheme that first predicts surface orientation and afterwards predicts the reflectance map by a learning-based sparse data interpolation. In order to analyze performance on this difficult task, wepropose a new challenge of Specular MAterials on SHapes with complex IllumiNation (SMASHINg) using both synthetic and real images. Furthermore, we show the application of our method to a range of image editing tasks on real images.