Abstract
In this work, we introduce a Hierarchical Generative
Model (HGM) to enable realistic forward eye image synthesis, as well as effective backward eye gaze estimation. The
proposed HGM consists of a hierarchical generative shape
model (HGSM), and a conditional bidirectional generative
adversarial network (c-BiGAN). The HGSM encodes eye geometry knowledge and relates eye gaze with eye shape, while
c-BiGAN leverages on big data and captures the dependency
between eye shape and eye appearance. As an intermediate component, eye shape connects knowledge-based model
(HGSM) with data-driven model (c-BiGAN) and enables
bidirectional inference. Through a top-down inference, the
HGM can synthesize eye images consistent with the given
eye gaze. Through a bottom-up inference, HGM can infer
eye gaze effectively from a given eye image. Qualitative and
quantitative evaluations on benchmark datasets demonstrate
our model’s effectiveness on both eye image synthesis and
eye gaze estimation. In addition, the proposed model is
not restricted to eye images only. It can be adapted to face
images and any shape-appearance related fields