arxivst stuff from arxiv that you should probably bookmark

Realistic Fake Data: How to Generate Images of Specific Objects

Post · Mar 30, 2017 16:54 ·

VAE GAN

GANs (general adversarial networks) are useful because they can generate images of flowers or faces, and this latest refinement can generate images of specific flowers or faces (e.g. a daisy instead of just a “pink blobby thing with petals”). In order to do that, the authors combine a GAN and a VAE (variational autoencoder) and throw in some interesting techniques (asymmetric loss and a mapping between latent and real space) to get it all to work. Based on what we saw in the paper, the network generates solidly realistic samples with high granularity.

Arxiv Abstract

  • Jianmin Bao
  • Dong Chen
  • Fang Wen
  • Houqiang Li
  • Gang Hua

We present variational generative adversarial networks, a general learning framework that combines a variational auto-encoder with a generative adversarial network, for synthesizing images of fine-grained categories, such as faces of a specific person or objects in a category. Our approach models an image as a composition of label and latent attributes in a probabilistic model. By varying the fine-grained category label fed to the resulting generative model, we can generate images in a specific category by randomly drawn values on a latent attribute vector. The novelty of our approach comes from two aspects. Firstly, we propose to adopt a cross entropy loss for the discriminative and classifier network, but a mean discrepancy objective for the generative network. This kind of asymmetric loss function makes the training of the GAN more stable. Secondly, we adopt an encoder network to learn the relationship between the latent space and the real image space, and use pairwise feature matching to keep the structure of generated images. We experiment with natural images of faces, flowers, and birds, and demonstrate that the proposed models are capable of generating realistic and diverse samples with fine-grained category labels. We further show that our models can be applied to other tasks, such as image inpainting, super-resolution, and data augmentation for training better face recognition models.

Read the paper (pdf) »