Abstract We introduce a new network structure for decomposing an image into its intrinsic albedo and shading. We treat it as an image-to-image transformation problem and explore the scale space of the input and output. By expanding the output images (albedo and shading) into their Laplacian pyramid components, we develop a multi-channel architecture that learns the image-to-image transformation function in successive frequency bands in parallel, within each channel is a fully convolutional neural network. This network architecture is general and extensible, and has demonstrated excellent performance on the task of intrinsic image decomposition. We evaluate the network on two benchmark datasets: the MPI-Sintel dataset and the MIT Intrinsic Images dataset. Both quantitative and qualitative results show our model delivers a clear progression over state-of-the-art