资源论文IRC-GAN: Introspective Recurrent Convolutional GAN for Text-to-video Generation

IRC-GAN: Introspective Recurrent Convolutional GAN for Text-to-video Generation

2019-10-08 | |  41 |   35 |   0

Abstract Automatically generating videos according to the given text is a highly challenging task, where visual quality and semantic consistency with text are two critical issues. In existing methods, when generating a specifific frame, the information in those frames generated before is not fully exploited. And an effective way to measure the semantic consistency between videos and given text remains to be established. To address these issues, we present a novel Introspective Recurrent Convolutional GAN (IRC-GAN) approach. First, we propose a recurrent transconvolutional generator, where LSTM cells are integrated with 2D transconvolutional layers. As 2D transconvolutional layers put more emphasis on the details of each frame than 3D ones, our generator takes both the defifinition of each video frame and temporal coherence across the whole video into consideration, and thus can generate videos with better visual quality. Second, we propose mutual-information introspection to semantically align the generated video to text. Unlike other methods simply judging whether the video and the text match or not, we further take mutual information to concretely measure the semantic consistency. In this way, our model is able to introspect the semantic distance between the generated video and the corresponding text, and try to minimize it to boost the semantic consistency. We conduct experiments on 3 datasets and compare with state-ofthe-art methods. Experimental results demonstrate the effectiveness of our IRC-GAN to generate plausible videos from given text

上一篇:iDev: Enhancing Social Coding Security by Cross-platform User Identification Between GitHub and Stack Overflow

下一篇:MAT-Net: Medial Axis Transform Network for 3D Object Recognition

用户评价
全部评价

热门资源

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...

  • Rating-Boosted La...

    The performance of a recommendation system reli...