Abstract Single-view intrinsic image decomposition is a highly ill-posed problem,and so a promising approach is to learn from large amounts of data.However;it is difficult to col- lect ground truth training data at scale for intrinsic images. In this paper, we explore a different approach to learning intrinsic images:observing image sequences over time de- picting the same scene under changing illumination,and learning single-view decompositions that are consistent with these changes.This approach allows us to learn without ground truth decompositions,and to instead exploit infor- mation available from multiple images when training.Our trained model can then be applied at test time to single views. We describe a new learning framework based on this idea, including new loss fiunctions that can be efficiently evaluated over entire sequences.While prior learning-based meth- ods achieve good performance on specific benchmarks,we show that our approach generalizes well to several diverse datasets,including MIT intrinsic images,Intrinsic Images in the Wild and Shading Annotations in the Wild