Abstract
Hyperspectral reconstruction from RGB imaging has recently achieved significant progress via sparse coding and
deep learning. However, a largely ignored fact is that existing RGB cameras are tuned to mimic human trichromatic
perception, thus their spectral responses are not necessarily optimal for hyperspectral reconstruction. In this paper,
rather than use RGB spectral responses, we simultaneously
learn optimized camera spectral response functions (to be
implemented in hardware) and a mapping for spectral reconstruction by using an end-to-end network. Our core
idea is that since camera spectral filters act in effect like
the convolution layer, their response functions could be optimized by training standard neural networks. We propose
two types of designed filters: a three-chip setup without spatial mosaicing and a single-chip setup with a Bayer-style
2x2 filter array. Numerical simulations verify the advantages of deeply learned spectral responses compared to existing RGB cameras. More interestingly, by considering
physical restrictions in the design process, we are able to
realize the deeply learned spectral response functions by
using modern film filter production technologies, and thus
construct data-inspired multispectral cameras for snapshot
hyperspectral imaging.