Abstract. Most conventional photometric stereo algorithms inversely
solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on
the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo
input and surface normals of a scene. For handling unordered, arbitrary
number of input images, we merge all the input data to the intermediate
representation called observation map that has a fixed shape, is able to
be fed into a CNN. To improve both training and prediction, we take
into account the rotational pseudo-invariance of the observation map
that is derived from the isotropic constraint. For training the network,
we create a synthetic photometric stereo dataset that is generated by a
physics-based renderer, therefore the global light transport is considered.
Our experimental results on both synthetic and real datasets show that
our method outperforms conventional BRDF-based photometric stereo
algorithms especially when scenes are highly non-convex